Exercise Booklet 2 final - Emergency Preparedness...Booklet 2: Selecting the Right Exercise!!!! A “How To” Series of Exercise Design, Conduct, and Evaluation ... A seminar is a

UC Berkeley Center for Infectious Diseases and Emergency Readiness 1

Public Health and Medical Exercise Planners’ Series

Booklet 2: Selecting the Right Exercise

!

! !!

A “How To” Series of Exercise Design, Conduct, and Evaluation Developed to Assist Public Health and Medical Organizations


Acknowledgements

Funding for this booklet was made possible through the Centers for Disease Control and Prevention (CDC) Grant 1 PO1 TP000295-01. This project was developed and completed by the Emergency Management Sciences Program (EMSci) of the University of California, Berkeley, Center for Infectious Diseases and Emergency Readiness (CIDER).

CIDER is one of three Centers for Public Health Preparedness in California and has the specific vision and mission to achieve “all-hazards public health and community emergency readiness through the provision of education and training, engaging in research activities which result in improved public health systems capabilities, increased local communities resilience to prevent, protect against, respond to, and recover from any public health emergency, and to effectively protect and serve our most vulnerable populations.”

The CIDER EMSci Program was created to assist public health and its partners, including city, county, State, and Tribal communities, with their education and training related to emergency management planning and implementation, and exercise design and evaluation. The EMSci Program utilizes the Homeland Security Exercise and Evaluation Program (HSEEP) to develop emergency management exercises that evaluate systems, provide opportunities to define gaps and vulnerabilities, and provide direction that will improve an agency’s operational readiness.

Contributors

The following UC Berkeley CIDER personnel contributed to this booklet:

Michael Petrie, EMT-P, MBA, MA, DirectorEmergency Management Sciences Program

Jeannie BalidoEducation and Training Program Manager

For more information please contact us at: www.idready.org

http://www.idready.org



Table of Contents

Acknowledgements ........................................................................................................ 2Contributors ................................................................................................................... 2

Introduction to the Public Health and Medical Exercise Planners’ Series ................ 4Purpose of the Series .................................................................................................... 4

Booklet 1: Introduction to Exercises .......................................................................... 4Booklet 2: Selecting the Right Exercise..................................................................... 4Booklet 3: Exercise Forms and Reference Documents............................................. 4

BOOKLET 2: Choosing the Right Exercise.................................................................. 6What the Exercise Planner Needs to Know ...................................................................... 6

Objectives...................................................................................................................... 6

Types of Exercises .......................................................................................................... 7Discussion-Based Exercises ......................................................................................... 7

Seminars ................................................................................................................... 8Workshops................................................................................................................. 8Tabletop Exercises .................................................................................................... 8Games ....................................................................................................................... 9

Operations-Based Exercises ......................................................................................... 9Drills .......................................................................................................................... 9Functional Exercises ............................................................................................... 10Full-Scale Exercises ................................................................................................ 10

Exercise Characteristics............................................................................................... 11

Selecting the Proper Exercise ..................................................................................... 12

Exercise Case Examples .............................................................................................. 13Example 1: Using Exercises to Develop and Validate Public Health Capabilities ....... 13Example 2: The Importance of Proper Exercise Selection .......................................... 19

Exercise Reference Documents................................................................................... 21


Introduction to the Public Health and Medical Exercise Planners’ Series

Purpose of the SeriesThe purpose of the Public Health and Medical Exercise Design Planners’ Series is to provide guidance, direction, and suggestions to public health or medical exercise planners and to introduce the Homeland Security Exercise and Evaluation Program (HSEEP) to the public health and medical community by demonstrating the use of that program throughout the booklet series. More than being another HSEEP guidance book, the Public Health and Medical Exercise Design Planners’ Series provides guidance for individuals and teams tasked with advancing preparedness in their organizations. These booklets provide an insider’s perspective, through the lens of an emergency planner, in the implementation of exercise activities from concept to conclusion for the novice to the more advanced exercise planner. There are three booklets in this series:

Booklet 1: Introduction to Exercises

Booklet 1: Introduction to Exercises introduces a five-phase approach to exercise foundation, design and development, conduct, and evaluation and improvement. This booklet provides direction and the “how to” in the design, development, conduct, and evaluation of public health and medically-focused emergency preparedness exercises.

Booklet 2: Selecting the Right Exercise

Booklet 2: Selecting the Right Exercise familiarizes the user with the seven standardized types of exercises in the US Department of Homeland Security’s Homeland Security Exercise Program (HSEEP). This booklet also identifies the key characteristics of each type of exercise, and presents a flowchart to help the user to select the proper

exercise. This booklet provides two case studies to demonstrate the use of exercises to develop and verify public health capabilities and to stress the importance of proper exercise selection and design.

Booklet 3: Exercise Forms and Reference Documents

Booklet 3: Exercise Forms and Reference Documents contains links to key exercise documents, templates, and forms used in the HSEEP exercise design, conduct, and evaluation process. Booklet 3 also presents two exercise scenarios, one discussion-based and one operations-based, which emphasize the documents used in exercise design, conduct, and evaluation. Booklet 3 provides valuable information on the Lessons Learned Information Sharing web portal, which is another valuable tool for exercise planners.


Objectives

After reviewing this booklet, the learner will have the basic knowledge and information to be able to:

1. Define the differences between discussion-based and operations-based exercises.2. Discuss the purposes and differences among the seven types of preparedness

exercises.3. Identify the key characteristics to consider when selecting an exercise.4. Determine how to select an exercise to meet the organization’s need.

Tips

Throughout the series, look for these unique indicators for tips and warnings:

Red Flag Warnings

Key Point

Lessons Learned

! Capability Measurement

This booklet is designed to assist exercise planners to design, conduct, and evaluate public health emergency management exercises to demonstratively improve public health preparedness. It is not intended to serve as a substitute for formal exercise design, conduct, and evaluation training, but as a supplement and reference during exercise planning activities.


What the Exercise Planner Needs to Know

Exercise design, conduct, and evaluation can be intimidating. It has a specific and evolving doctrine. It has its own terminology and acronyms like SITMAN, EXPLAN, AAR, EEG, MSEL and HSEEP. It has forms that are daunting and redundant to the untrained. Yet exercising is a critical component of public health and medical preparedness, as it is for all disciplines that have responsibilities to protect the public’s safety, security or health. Exercises are an essential tool to create and plan preparedness concepts; to develop and revise policies, procedures, and practices; and to evaluate personnel, equipment, and systems.

This booklet (number 2 in a 3-part series), familiarizes the user with the seven standardized types of exercises in the US Department of Homeland Security’s Homeland Security Exercise Program (HSEEP). This booklet also identifies the key characteristics of each type of exercise, and presents a flowchart to help the user to select the proper exercise.

For most organizations with preparedness responsibilities, including public health, emergency medical services, and healthcare organizations, compliance with the HSEEP program is essential for two reasons. First, it is a structured program, which if followed properly, can lead to demonstrative improvements in preparedness. Second, in many instances, eligibility for federal and state grants is contingent upon participation in HSEEP-compliant exercises.

This booklet series is not a substitute for formal exercise training. UC Berkeley CIDER strongly encourages all exercise planners to receive formal training in the HSEEP exercise program.

Booklet 2: Choosing the Right Exercise


The Homeland Security Exercise and Evaluation Program (HSEEP) identifies seven distinct types of exercises. Four of these are discussion-based exercises, which are exercises where is no movement of field assets. The discussion exercises are seminars, workshops, tabletops, and games. The remaining three HSEEP exercises are operations-based exercises. These exercises are characterized by time-indexed reaction of players to information inputs, and often there is movement of resources in a field setting. The three operations-based exercises are drills, functional exercises, and full scale exercises. Each type of exercise has specific purposes, characteristics, benefits, and limitations.

Those responsible for designing, conducting, and evaluating exercises should not fixate on a specific type of exercise. The exercise leaders should select the proper exercise from the full range of exercises to help their organization identify issues; develop policies, procedures, and practices; and validate capabilities. The definition, characteristics, benefits and limitations of each type of exercise are discussed in the following pages.

Discussion-Based Exercises

Discussion-based exercises typically highlight existing plans, policies, interagency agreements, and procedures. Discussion-based exercises are used for collaborative problem-solving; development or orientation of staff to a policy or procedure; or evaluation of the effectiveness and appropriateness of a policy, procedure or plan. Discussion-based exercises focus on strategic, policy-oriented issues. These exercises are typically coordinated by a facilitator or presenter.

Types of Exercises

Discussion-based exercises are used for planning. They cannot be used to validate an organization’s capabilities. Capabilities can only be evaluated through operations-based exercises.


Seminars

Seminars are meetings used to orient participants to strategies, plans, policies, or concepts and ideas. A seminar is a good starting point for agencies that are developing or making substantial changes to their plans and procedures. One or more presenters provide information to an audience, without significant two-way communication. They are usually led by a presenter, but may be augmented by a panel or a subject matters expert. Seminars are informal discussions not constrained by real-time portrayal of events. Seminars are low stress events.

Workshops

Workshops are the second tier of exercises in the HSEEP building-block approach. A workshop is used to develop a specific product, such as a policy or procedure, and is often used to secure feedback after the first draft of a policy and procedure has been written. Workshops are similar to seminars except that communication between the facilitator and participants is increased. In workshops, participants provide their perspectives and feedback on the policy or topic the facilitator is presenting. Workshops are low stress events that are not time constrained.

Tabletop Exercises

Tabletop Exercises are the third tier of exercises in the HSEEP building block approach. Tabletop exercises are commonly used to evaluate plans, policies, procedures or systems. During tabletop exercises, exercise players discuss their organization’s response or other activities, guided by a scenario. Exercise players discuss issues in detail, such as the real impacts of using a certain policy or practice during an emergency response. Exercise play can be stopped or slowed to discuss potential problems or best practices in detail. Tabletop exercises are typically low stress events held in an informal environment.

There are two types of tabletop exercises, basic and advanced. In a basic tabletop exercise, the scenario remains as written in the situation manual (SITMAN), regardless of the decisions made by the exercise players. To an extent, decisions made early in exercise play do not affect play later in the exercise.

In an advanced tabletop exercise, messages injected into exercise play change the scenario throughout the exercise. Players respond to each message or inject, using the policies, procedures and practices of that agency. In some advanced tabletop exercises, decisions made early in play effect later exercise play.

A tabletop exercise could be used to asses the feasibility of a multi-causality incident response policy in an EMS System. The situation and victim status might be described, and each hospital and ambulance provider would answer specific about their capabilities and procedures related to the scenario. Through this process, a common concept of operations might be created, and deficits that might otherwise be missed, might be identified for correction.


Games

A game is a computer or internet-based simulation involving two or more teams. Games provide realistic computer-generated scenarios, which are used to evaluate the players’ decision making process, based upon the scenarios and the organization’s policies and rules. One benefit of a game is that the consequences of prior decisions affect future game play. For example, if a county’s only epidemiological response team is sent away for mutual aid, that team can’t be later in the game. Because many games are Internet-based, geographically disparate teams can train or evaluate their performance without travel.

Operations-Based Exercises

Operations-based exercises are typically used to validate plans, policies, procedures, and systems. Operations-based exercises include drills, functional exercises and full-scale exercises. When properly selected and designed, these exercises are effective to clarify roles and responsibilities, identify gaps in resources needed to implement plans and procedures, and improve individual and team performance. Operations-based exercises are characterized by play in real time, and actual decision making or movement based upon a scenario. Often, the movement during operations-based exercises involves “boots on the ground”; the movement of field assets, such as vehicles and personnel.

Drills

A drill is a coordinated, supervised activity usually used to evaluate a single function in a single agency or department. Drills are usually used to train personnel on new equipment, develop or validate policies or procedures, or practice and maintain current skills. Drills provide the opportunity to evaluate one field function in a somewhat realistic environment, in real time. A good example of a drill is a fire drill from your child hood. It was a supervised (by your teacher) activity to validate one function (rapidly and safely exiting the classroom) in a single agency (your class or school).

You do not have to strictly follow the building block approach. Select the exercise that best meets your organization’s needs.

Drills are an effective way to verify capabilities of one specific function, such as victim decontamination. However, will the assets that support the decon team during the drill be available during a disaster or will those support assets have other responsibilities? At some point in the exercise

cycle, all critical skills should be verified as a component of a full scale exercise.


Functional Exercises

A functional exercise is used to validate and evaluate the capabilities of individuals and groups, and functions in a command or management environment. Response-focused functional exercises typically evaluate the ability of Incident Command, Unified Command, Emergency Operations Center, Department Operations Center or Multi-Agency Coordination Center managers. Functional exercises evaluate managers through a rapidly changing scenario designed to present complex incident problems that simulate operations within a command center. Functional exercises typically are held in an Incident Command Post, Department Operations Center or Emergency Operations Center. Communication with other operations centers or field assets are simulated by simulation cells. These exercises are stressful and conducted in real time. The functional exercise is the only operations-based exercise in which the movement of field personnel and equipment is simulated.

Full-Scale Exercises

The full scale exercise is the most complex type of exercise. Full scale exercises are typically multi-agency, multi-jurisdictional, multi-discipline exercises that simulate many elements of a complex incident response, and validate many aspects of preparedness. Full scale exercises focus on the implementation of policies, procedures and agreements by individuals and groups of people (similar to a functional exercise). These exercises also simultaneously focus on the deployment, coordination and control of personnel and equipment in a field setting (similar to a drill). However, full scale exercises are multi-organizational and multidisciplinary; therefore, they are similar to numerous drills being coordinated from a single incident or unified command post.

Full scale exercises feature simultaneous operations in many functional areas that present complex problems. These problems require critical thinking, rapid decision-making, and effective responses by trained personnel and their equipment. Full scale exercises are held in real time and are stressful. Full scale exercises are complex; the largest full scale exercises last many days, involve dozens of federal, state, and local agencies, and take years to plan.

Full scale exercises should only be used to validate capabilities after component capabilities, such as communications, command and coordination, and policies and procedures have been verified through other less resource-intensive exercises.


Exercise Characteristics

The following table identifies the key characteristics of the each of the seven types of exercises identified in the Homeland Security Exercise and Evaluation Program (HSEEP).


Selecting the Proper Exercise

It is critical to select the proper exercise to meet the needs of your organization. Selecting the wrong type of exercise can be extremely expensive, fail to meet your organization’s planning needs, or provide erroneous validation of an organization’s capabilities. The following flowchart will help you select the proper exercise to: 1) to develop or revise a policy, procedure, agreement or system; 2) to evaluate policies, procedures and systems; and, 3) to validate your organization’s capabilities. The flowchart does not consider grant requirements for a specific type of exercise, grant timelines for exercises, and other grant program or HSEEP requirements. Grant guidance may limit how

exercise funding may be spent.!


This section contains two case examples. The first case example describes in detail, how public health organizations can use exercises to develop and validate public health capabilities. The second example identifies the importance of proper exercise selection, and demonstrates how selecting the wrong type of exercise can produce misleading results.

Example 1: Using Exercises to Develop and Validate Public Health Capabilities

This example illustrates how a public health department can use HSEEP exercises to develop the policies and procedures for a multi-jurisdictional epidemiological field team. This example also illustrates how a health department can use HSEEP exercises to validate the capabilities of a multi-jurisdictional epidemiological field team.

The CIDER County Health Department wants to coordinate the development of a multi-county epidemiological field team to conduct outbreak investigations. CIDER County and neighboring counties would each provide their trained personnel to support this multi-county team.

The CIDER County Director of Health‘s vision is that during a public health outbreak or emergency, an impacted county could request epidemiological field team assistance from any of the other participating counties. These field team members will have common training, work under common policies and protocols, and use common communication systems. The Director of Health believes this provides participating counties with surge capacity and a better capacity to investigate diseases that cross county lines.

The Director of Health meets with the Health Department’s Emergency Preparedness Director. They determine that they would like to use an exercise-based process to develop the team and its policies. The Emergency Preparedness Director, having recently received Homeland Security Exercise and Evaluation Program (HSEEP) training, recommends the use of three exercises, following the HSEEP format. In the first exercise they will share their vision, gain support, and reach consensus for the multi-county epidemiological field team concept. In the second exercise, they will develop a policy and procedure for the multi-county team. In the third exercise, they will assess the draft policy and procedure through a simulated scenario.

To begin this process, the CIDER County Public Health Emergency Preparedness Director reaches out to representatives from each of the five health departments that will be asked to participate in the multi-county epidemiological field team emergency. These six preparedness directors will serve as the Exercise Planning Team. Together, they will design the exercises, lead the logistic coordination for the exercises, manage budgets and track costs for the exercises, and lead the evaluation and improvement planning processes after the exercise.

Exercise 1: Seminar The goal of the first meeting, or exercise, is to develop agreement on the need for a multi-county epidemiological field team, gain each county’s support to participate in the team, and agree on a process to develop the team. At the first exercise, the CIDER County Health Department and the exercise planning team will present the concept of the multi-county epidemiological field team to the leadership of the neighboring five health departments. The

Exercise Case Examples


CIDER County Health Director will lead an informal discussion on creating the team. This discussion will be augmented by a panel presentation by health department leaders who have successfully implemented multi-jurisdictional field teams. The health director will answer questions about the team concept, cost sharing, and coordination mechanisms. She will develop consensus, and describe the process to move forward toward implementing the team. This meeting or exercise is called a Seminar in homeland Security Exercise and Evaluation Program (HSEEP) terminology.

In this instance, all of the invited county health departments agreed to participate in the multi-county epidemiological field team, and agreed to an exercise-based development process.

Exercise 2: WorkshopThe next step in developing the multi-county epidemiological field is to develop policies and procedures, defining the process for alerting the team, and defining the process for coordinating the field team from the requesting-health department’s DOC. To achieve these goals, the CIDER Health Department will lead the other participating health department emergency preparedness representatives in a workshop. At the workshop, the CIDER Health Department Emergency Preparedness Coordinator will clearly describe the goals to be accomplished and the process the group will use at the workshop to achieve goals. The group will have a dialogue-rich discussion to identify each county’s specific requirements for their personnel to be alerted, and how these teams will be coordinated by the county leading the outbreak investigation. Representatives will share their perspectives, discuss how certain alternatives work for their department and personnel, and jointly craft the policies. While the facilitator leads the workshop, all representatives (exercise players) have considerable input into the interactive discussion.

At the end of the 8-hour workshop, the participating health departments have developed two draft policies. The first policy describes the process to alert the multi-county epidemiological field team and the second policy describes the processes used by the requesting county’s health department DOC to coordinate the epidemiological field teams.

Exercise 3: Tabletop ExerciseAfter reviewing the two draft policies created in the workshop, the CIDER County Health Director wanted to assure that the processes identified in the policies will work before they are implemented. She considers it prudent to assess the policies before incurring the expense and time commitments of printing the policy manual and training staff. She requests that the multi-departmental emergency planning committee conduct a table top exercise to assess the policy by testing it through a scenario-based exercise.

The multi-county emergency planning committee conducts two meetings to plan for the tabletop exercise. The first meeting occurs 3 months before the exercise and the second about 6 weeks before the exercise. At these meetings, the committee creates the exercise scenario and discussion questions, plans an evaluation mechanism, and writes the situation manual (SITMAN) which will be used by the exercise participants. They also plan for the use of visual aids, identify a facilitator and evaluators, and select a venue for the exercise.

The day of the exercise, the facilitator presents the scenario contained in the SITMAN to the employees who will work on the multi-county field epidemiology team. The facilitator provides background information and describes numerous patients presenting with SARS-like symptoms at area hospitals. Audio and video recordings make the scenario realistic. The facilitator guides discussion among the players, who describe their response to the scenario, based upon the draft policies and procedures. Limitations of the policies and discovered, discussed in depth, and new policy language is created. The exercise also identifies certain new equipment that will


be needed by the team. Changes to the policies and lists of new equipment are captured in the debriefing session that follows the exercise. In the days following the tabletop exercise, the multi-county emergency planning committee creates an After Action Report and Improvement Plan, which identifies the lessons learned from the exercise, the strengths and areas for improvements, and documents the steps that must be taken to finalize the policies for distribution.

Exercise 4: GameThe multi-county exercise planning committee explored whether a game would be valuable to the development of the policies. They understood that a game is a computer or internet-based simulation involving two or more teams, which are used to evaluate the players’ decision making process, based upon the scenarios and the organization’s policies and rules. However, they determined that they were ready to finalize the policies and staff the team, based on the results of their tabletop exercise, as documented in the After Action Report.

At this point, the team’s policies and procedures have been created using the HSEEP exercise process. During the same time, health department executives have developed a funding mechanism for the team, hired epidemiologists to staff the team, and purchased the team’s equipment. The team’s leaders have thoroughly trained their staff.

Before the team is placed “on line”, the Directors of Public Health want to evaluate the epidemiological field team’s capabilities through an exercise-based process. They will use three exercises for this purpose. The first exercise will test the ability of the field teams to respond to hospitals to interview patients and review patients’ charts. The second exercise will test CIDER County’s Health Department Operating Center (DOC) staff’s ability to coordinate epidemiological field teams. The third exercise will evaluate the ability of the CIDER County’s DOC to coordinate epidemiological field teams, while simultaneously coordinating other public health preparedness functions, including laboratory investigation and setting up Points of Dispensing (PODs).

Exercise 5: DrillThe multi-county exercise planning committee must conduct an exercise to test the ability of the field teams to respond to hospitals to interview patients and review patients’ charts, which will be a critical skill for the multi-county epidemiological field team. The planning committee determines that the best exercise for this purpose is a drill, which is a real-time coordinated activity to test a single function in a field setting.

The multi-county emergency planning committee conducts three meetings to plan for the drill. The first meeting occurs six months before the exercise. At this meeting, they define the exercise concept and objectives, begin to write the exercise plan (EXPLAN) and master scenario event list (MSEL) and reach out to other organizations, such as hospitals, that will participate in the exercise. The second meeting occurs 3 months before the exercise. At this meeting, they discuss exercise staffing requirements, scheduling and logistics, such as venue selection. They also identify actors who will portray the “patients”. The final meeting occurs about 6 weeks before the exercise. At this meting, the committee completes their final review of all exercises processes and procedures. They also complete their final drafts of all exercise documents.

The day of the exercise, the members of the multi-county epidemiological response team arrive at a simulated workplace. These team members will be the exercise players. Their performance will be assessed by an evaluator, who is assigned to each player. The exercise controller provides players with the scenario’s background. Each player is dispatched via radio to a local


hospital, where an actor simulates a patient. The players takes their assigned equipment, and based on the policy developed for the team, interviews a number of patients, reviews the patients’ charts, and notes key findings on the team’s assessment form. After collecting and recording this information, they return to the simulated workplace, completing the exercise.

The lead evaluator guides the players in a hotwash, which is a debriefing process. Later that day, the evaluators and controllers meet, review the exercise evaluation guides, which evaluate players against pre-identified policy standards. They also discuss their own observations of the players’ performance in the exercise. Limitations of the response are identified, and discussed in depth. For example, one radio used by a player had dead batteries and failed during the exercise. This type of problem was not identified in the earlier tabletop exercise. The exercise also identifies additional equipment that will be needed by the team.

In the days following the tabletop exercise, the multi-county emergency planning committee creates an After Action Report and Improvement Plan, which identifies the lessons learned from the exercise, the strengths and areas for improvements, and documents the steps that must be taken to verify the field team is ready to respond to a real incident.

Exercise 6: Functional ExerciseThe capabilities of the field team were validated during the drill. However, the ability of the CIDER County’s Health Department Operating Center (DOC) staff’s ability to coordinate epidemiological field teams has not yet been verified. The multi-county exercise planning committee determines that the best exercise to verify this capability is a functional exercise, which is a real time exercise used to validate individual or group functions, often within an emergency operations center or department operations center.

The multi-county emergency planning committee conducts three meetings to plan for the functional exercise. The first meeting occurs six months before the exercise. At this meeting, they define the exercise concept and objectives; begin to write the exercise plan (EXPLAN) and master scenario event list (MSEL). The second meeting occurs 3 months before the exercise. At this meeting, they discuss exercise staffing requirements, scheduling and logistics, such as venue selection, and identifying personnel who will work in the simulation cell (SIMCELL). SIMCELL personnel will simulate epidemiological field team members, who in reality will not participate in this exercise. The final meeting occurs about 6 weeks before the exercise. At this meting, the committee completes their final review of all exercises processes and procedures. They also complete their final drafts of all exercise documents.

The day of the exercise, key CIDER Health Department personnel who normally staff the Health Department Operating Center (DOC) report to the DOC. These personnel will be the exercise players. Their performance will be assessed by evaluators, who are assigned to each functional area in the DOC. In this exercise, the functional areas being tested are Command, Plans, Operations, and Logistics. The exercise controller provides players with the scenario’s background and begins the exercise. The evaluators asses the ability of this command and control team to create an incident response action plan, provide direction and information to simulated field teams, and provide for the resupply of the field teams, based on their training and guidance provided in the organization’s policies. SIMCELL personnel simulate hospitals, which report the number of victims in their facilities. SIMCELL personnel also simulate the field response teams. The exercise ends 4-hours later. They lead evaluator guides the players in a hotwash, which is a debriefing process. Later that day, the evaluators and controllers meet, review the exercise evaluation guides, which evaluate players against pre-identified policy standards. They also discuss their own observations of the players’ performance in the exercise. Performance limitations are identified, and discussed in depth. For example, no one on the DOC


team was familiar with the standardized Incident Command System (ICS) Incident Action Plan forms. This deficiency was recorded as an area for future training. In the days following the tabletop exercise, the multi-county emergency planning committee creates an After Action Report and Improvement Plan, which identifies the lessons learned from the exercise, the strengths and areas for improvements, and documents the steps that must be taken to verify that the DOC team is able to coordinate the actions of the epidemiological field team during a real incident.

At this point, both the DOC team and the epidemiological field response teams have demonstrated their ability to perform their respective functions. The deficiencies identified in the drill and functional exercises have been corrected in each team, through additional training, equipping and organizing. It is now time to verify and validate the capabilities of the complete system, consisting of the DOC team and the epidemiological response team. For this purpose a full scale exercise will be used. A full scale exercise was not appropriate earlier in the epidemiological field team development or verification process, as the cost, time, and other resources used in a full scale exercise is considerably more than in all of the other types of exercises. Additionally, the drill and the functional exercise identified numerous deficits, which were resolved, before the full scale exercise.

Exercise 7: Full Scale ExerciseThe capabilities of the field team were validated during the drill, and the capabilities of the CIDER County’s Health Department Operating Center (DOC) staff were validated during the functional exercise. However, these exercises tested those organizations’ capabilities in isolation. There were no competing demands on the field team or the DOC staff. To make the full scale exercise as real as possible, the multi-county emergency planning team will incorporate competing functions that may occur during a real-world infectious disease outbreak. The functions to be incorporated and evaluated include CIDER County’s DOC’s ability to coordinate epidemiological field teams, while simultaneously coordinating laboratory investigations and setting up Points of Dispensing (PODs).

Due to the complexity of this full scale exercise, the multi-county emergency planning committee conducts four meetings to plan for the full scale exercise. The first meeting occurs one year before the exercise. At this meeting, a concept and objectives meeting, they identify exercise goals, identify exercise team members, and identify what capabilities will be tested. The second meeting, the initial planning conference, occurs 9 months before the exercise date. At this meeting, team members focus the scope of the exercise, identify the venues for the exercise, and discuss other assumptions and artificialities. Team members are assigned to write the exercise plan (EXPLAN) and master scenario event list (MSEL). The third meeting, the midterm planning conference, occurs 3 months before the exercise. At this meeting, they discuss exercise staffing requirements, scheduling and logistics issues. They identify personnel who will work in the simulation cell (SIMCELL), simulating non-participating entities. SIMCELL personnel will simulate epidemiological field team members, who in reality will not participate in this exercise. They also review the progress on the EXPLAN, MSEL, and other documents. The fourth meeting, the final planning conference, occurs about 6 weeks before the exercise. At this meting, the committee completes their final review of all exercises processes and procedures. They also complete their final drafts of all exercise documents.

The day of the exercise, exercise controllers and evaluators arrive at the county center, which is the primary exercise site. The exercise players also arrive. The exercise players include the members of the multi-county epidemiological field team, the DOC staff, those assigned to open points of distribution (PODs), and laboratory personnel. The lead controller explains the exercise, provides safety information, briefs on the current exercise status, and begins the


exercise. Based on information provided through injects from the SIMCELL, the DOC staff are activated. To investigate the cases, they deploy numerous field epidemiological field teams to area hospitals. The field teams interview patients and review their charts. Biological samples are provided to the laboratory personnel for analysis. Early information from the laboratory and the field teams suggest the presence of a highly infectious and lethal disease in the community. Based on this information, the CIDER County Director of Health and other officials order that PODs be opened for mass vaccination. The DOC staff coordinates the deployment, logistical support and security of the PODs. Throughout the course of the 8 hour exercise, the DOC staff, the epidemiological field teams, the laboratory staff and the POD staff are evaluated through a changing scenario.

At the end of the day, the lead evaluator guides the players in a hotwash, which is a debriefing process. The next day, the exercise evaluators and controllers meet; review the exercise evaluation guides, which evaluate players against pre-identified policy standards. They also discuss their own observations of the players’ performance in the exercise. Limitations of the response are identified, and discussed in depth. For the most part, limitations occurred in coordinating multiple teams, which were competing for scare resources. In the days following the tabletop exercise, the multi-county emergency planning committee creates an After Action Report and Improvement Plan, which identifies the lessons learned from the exercise, the strengths and areas for improvements, and documents the steps that must be taken to assure that the CIDER County Health Department and the multi-county epidemiological field team is ready to respond to a real incident.


Example 2: The Importance of Proper Exercise Selection

A jurisdiction’s hazardous materials (HazMat) team wanted to test their ability to decontaminate 500 victims per hour, following a CBRN (Chemical, Biological, Radiological, or Nuclear) incident. They tested this capability using a drill in which the HazMat team decontaminated six ambulatory “victims”. In this drill, support team members erected the decontamination tent while HazMat members donned Level A suits. The chemical agent was pre-detected and identified, thus, HazMat team members did not have to perform this function. Using a megaphone, a team member directed the contaminated victims to disrobe to their swim shorts, walk through fire department’s hose streams and then walk through and scrub themselves in a decontamination tent. All of the six, prescreened, pre-trained, ambulatory, healthy, English-speaking, order-compliant victims with good hearing performed the tasks as directed. At the end of the exercise, the jurisdiction’s chief executive boasted of the jurisdiction’s robust victim decontamination capability. No exercises were planned within the next five years to further test this capability.

From the information provided, what were the problems with this exercise’s design, conduct, and evaluation, and what are the consequences of those problems?

The design and conduct of the exercise did not sufficiently approximate the real-world conditions for which the organization is preparing.

To reliably validate an organization’s capabilities, an exercise, or an exercise in a series of exercises, must sufficiently approximate the real-world conditions for which the organization is preparing. Decontaminating six victims does not sufficiently approximate the complexity of decontaminating 500 victims. Decontaminating 500 victims will require more HazMat team members to perform victim decontamination. Team members will have to cycle to decontaminate victims, rotating as they fatigue or run short of air. They will have to enter and exit their Level A suits, being decontaminated before exiting the suit. Personnel will out of service as they rehabilitate after working in the Level A suits. Because of the number of HazMat personnel involved, larger, more complicated support and command and control personnel are required. Additionally, more resources, such as HazMat suits, roller transport trays, air bottles, water, hose, decontamination tents, privacy areas, modesty covers, and radios will be required.

The victims in this scenario are not similar to the victims a jurisdiction would encounter in a real incident. Following a 500 victim CBRN incident, many victims will be not be ambulatory. Many will be carried on stretchers or gurneys, complicating the decontamination process. Few will be healthy. Victims may be hard of hearing, non-English speaking, or non-order-compliant. Some may avoid the decontamination process due to concerns of modesty. Again, the victims in this exercise are not representative of the victims expected in the incident for which the jurisdiction was preparing.

A drill is useful to evaluate the capability of one field team to perform one field function. However, during a CBRN incident, victim decontamination is one of many critical functions that most be rapidly and simultaneously performed. Victims must be evacuated from the contaminated area (hot zone); exigent life-saving medical treatment must be provided; victims must be transported to hospitals or alternate care sites. Each of these critical functions will be resource competitive—that is they will complete for scare personnel and equipment. Often these functions are performed by the same organization, and that organization’s ability to intensively perform one critical function may degrade or limit their ability to perform other functions. The drill described in this scenario did not consider scarcity of personnel or equipment. The drill also did not evaluate the ability of the HazMat team to decontamination victims, while members of that


same team rescue victims from the hot zone, provide exigent medical care in the warm zone, and prepare victims for transportation to the hospital. These competing demands certainly will affect the HazMat team’s ability to provide victim decontamination.

This drill, as designed, may have been beneficial if it was one drill in a series of exercises, leading to a full scale exercise that sufficiently approximated a 500-victim CBRN incident. But, this was a stand alone drill and no follow on full scale exercise was planned or conducted. At some point in an exercise series, a full scale exercise is necessary to validate an organization’s ability to perform and control the numerous, simultaneous, resource-competitive functions that are expected during a critical incident or disaster.

One detrimental consequence of improper exercise selection, such as the use of an isolated drill to verify a HazMat team’s victim decontamination capability, is that a poorly selected exercise may suggest capabilities that do not exist or have not yet been verified. This is particularly detrimental if the inaccurate assessment dissuades future training, exercises, improvements, or other preparedness activities.


The following documents were used during the development of the Public Health and Medical Planner’s Exercise Series. These reference documents, listed alphabetically, define and describe the Homeland Security Exercise and Evaluation Program (HSEEP), and provide essential background regarding incident management systems, identifying target capabilities, and planning using standardized scenarios:

California Homeland Security Exercise and Evaluation Program (HSEEP) Training Course, California Emergency Management Agency (CalEMA).

Enhanced Exercise Design, Conduct and Evaluation (EEDCE) Participant Guide, California Emergency Management Agency (CalEMA), California Specialized Training Institute (CSTI)

Enhanced Exercise Design, Conduct and Evaluation (EEDCE) Additional Resource Book, California Emergency Management Agency (CalEMA), California Specialized Training Institute (CSTI)

Emergency Management Institute, Independent Study Course Workhttp://training.fema.gov/ IS-120A – Introduction to Exercises IS-130 – Exercise Evaluation and Improvement Planning IS-139 – Exercise Design

Homeland Security Exercise and Evaluation ProgramExercise Manuals 1 through 4https://hseep.dhs.gov

Homeland Security Exercise and Evaluation Program (HSEEP) Quick Reference Guide, UC Berkeley CIDER www.idready.org

National Incident Management System, December 2008 http://www.fema.gov/pdf/emergency/nims/nims_doc_full.pdf

National Planning Scenarios, March 2006 (located in secure portal)http://www.llis.dhs.gov

National Preparedness Guideline, September 2007http://www.dhs.gov/xlibrary/assets/National_Preparedness_Guidelines.pdf

National Response Framework, January 2008http://www.fema.gov/pdf/emergency/nrf/nrf-core.pdf

Target Capabilities List, September 2007 http://www.fema.gov/pdf/government/training/tcl.pdf

Universal Task List, February 2007 (located in secure portal)http://www.llis.dhs.gov

Exercise Reference Documents

http://training.fema.gov/

http://training.fema.gov/

https://hseep.dhs.gov

https://hseep.dhs.gov



http://www.fema.gov/pdf/emergency/nims/nims_doc_full.pdf

http://www.fema.gov/pdf/emergency/nims/nims_doc_full.pdf

http://www.llis.dhs.gov


http://www.dhs.gov/xlibrary/assets/National_Preparedness_Guidelines.pdf

http://www.dhs.gov/xlibrary/assets/National_Preparedness_Guidelines.pdf

http://www.fema.gov/pdf/emergency/nrf/nrf-core.pdf

http://www.fema.gov/pdf/emergency/nrf/nrf-core.pdf

http://www.fema.gov/pdf/government/training/tcl.pdf

http://www.fema.gov/pdf/government/training/tcl.pdf

