Homeland Security Exercise Evaluation Programwyohomelandsecurity.state.wy.us/training/L0146 HSEEP...Module 1: Introduction and HSEEP Fundamentals L0146 Version 13.1 HSEEP Training

Module 1: Introduction and HSEEP FundamentalsL0146 Version 13.1

Homeland Security Exercise Evaluation Program

INTRODUCTION & FUNDAMENTALS

1


HSEEP Training Course AgendaDay One Module 1 – Introduction and Fundamentals

Module 2 – Program Management

Module 3 – Design and Development

Day Two Module 3 – Design and Development

Module 4 – Exercise Conduct

Module 5 – Exercise Evaluation

Module 6 – Improvement Planning

2


Course Overview & Target AudiencesThe target audience for HSEEP training includes:• Exercise Planning Team Members

• Exercise Controllers and Evaluators

• Exercise Facilitators

• State Administrative Agency (SAA) Exercise Program Managers

• Elected and Appointed Officials

• Exercise Players

• Anyone Expected to Participate in an Exercise Program

3


Course Materials and Jurisdictions• At your tables, you will find: Participant Handouts

► Course Pack with all the slides► Activity Booklet► Mission Areas and Core Capabilities► The HSEEP manual in booklet format

Dry Erase Table Tent Jurisdictional Designation

► City or County in the State of Columbia

4


IntroductionsInstructor Introductions

Participant Introductions—please respond with:• Name preference

• Agency/Organization/Affiliation

• Previous exercise experience

• Course expectations

5


Canada

Roaring RiverTribal Community

State of Columbia


Module 1: Introduction to HSEEPLesson 1: HSEEP Overview• Purpose• Applicability and ScopeLesson 2: HSEEP Fundamentals• HSEEP Guiding Principles• Key Program Elements• HSEEP Methodology

7


HSEEP PurposeThe Homeland Security Exercise and Evaluation Program (HSEEP) provides a set of guiding principles for exercise programs, as well as a common approach to exercise program management, design and development, conduct, evaluation, and improvement planning.

8


What Is HSEEP?A consistent approach to capabilities-based exercise program management that uses a common methodology to measure progress toward building, sustaining, and delivering core capabilities.

9


Why Exercise?• Test and Validate Plans

Capabilities

• Identify Resource Requirements

Capability Gaps

Areas for Improvement

10


HSEEP Guidance• HSEEP Fundamentals• Program Management• Design and Development• Exercise Conduct• Exercise Evaluation• Improvement Planning

11


HSEEP Applicability and ScopeFlexible, scalable, adaptable for use by… • Stakeholders across Whole Community• Applicable to exercises for all Mission Areas: Prevention Protection Mitigation Response Recovery

12


Lesson 2: HSEEP FundamentalsEnabling Objectives After completing this lesson, you should be able to:• Describe the HSEEP methodology as set forth in

the exercise planning cycle• Identify how the HSEEP principles integrate

fundamental principles of the National Preparedness System (NPS)

13


HSEEP Fundamental Principles

• Guided by Elected and Appointed Officials• Capability-Based and Objective-Driven• Utilizes Progressive Planning Approach• Whole Community Integration• Informed by Risk• Common Methodology

14


Common Methodology

15


Presidential Policy Directive 8 (PPD-8)• National Preparedness System• National Preparedness Goal

16


Lesson 2 ReviewIn this lesson we have:• Described the HSEEP methodology as set forth

in the exercise planning cycle• Identified how the HSEEP principles integrate

fundamental principles of the National Preparedness System (NPS)

17


Module 1: SummaryIn this module, we have discussed:

• How HSEEP addresses National Preparedness by incorporating Whole Community Principles to facilitate self-sustaining exercise programs

• HSEEP methodology and capabilities-based planning

Module 2: Exercise Program

Management

18


Module 2HSEEP PROGRAM MANAGEMENT

19


Module 2: Agenda

Lesson 1: Engaging Elected and Appointed Officials

Lesson 2: Multi-year Training and Exercise Planning

20


Overview of Exercise Program Management

21


Lesson 1: Engaging OfficialsEnabling ObjectivesAfter completing this lesson, you should be able to identify:• How components of the National Preparedness System

provide a foundation for identifying training and exercise program priorities

• The role elected and appointed officials play in identifying priority capabilities that will be addressed through training and exercises

• The importance of engaging these officials early and often throughout the HSEEP program management process

22


National Preparedness System Components

23

1. Identify and Assess Risk

2. Estimate CapabilityRequirements

3. Building and Sustain Capabilities

4. Plan to Deliver Capabilities

5. Validate Capabilities

6. Review and Update Risks, Tools, and Resources


Exercise FoundationLearn about your jurisdiction:• Threats, Hazards, and

Vulnerabilities

• Policies, Plans, and Procedures

• Grant or Cooperative Agreements

• Multi-year Training and Exercise Plans (TEP)

• After-Action Reports/ Improvement Plans (AAR/IP)

• Identified needs—Training, Equipment, Personnel

24


Elected and Appointed Officials• Engage early and often to identify exercise

program priorities• Review previous risk assessments and reports• Provide: Overarching guidance and direction

Specific intent for individual exercises

25


Multi-year Program Priorities• Identified by elected and appointed officials• Guide program planning and resource allocation• Determine type and range of training & exercises• Provide a roadmap for selecting and prioritizing

individual exercise development

26


Lesson 1: ReviewIn this lesson, we have discussed:• How components of the National Preparedness System

provide a foundation for identifying exercise program priorities

• The role elected and appointed officials play in identifying priority capabilities that will be addressed through training and exercises

• The importance of engaging these officials early and often throughout the HSEEP program management process

27


Lesson 2: Developing a multi-year Training and Exercise Plan Enabling ObjectivesAfter completing this lesson you should be able to explain:• The purpose of a Training and Exercise Planning

Workshop (TEPW)• The purpose a multi-year Training and Exercise Plan

(TEP) and development and coordination of the multi-year Training and Exercise Schedule

• Exercise Program Manager’s role in managing program resources and how the Rolling Summary Report supports these efforts

28


Training & Exercise Planning Workshop• Program-Level Planning• Held on a periodic basis• Planning based on program needs or pre-

determined requirements

29


Why? Purpose of the TEPW• Translate priorities into

specific objectives, training and exercises

• Track IP actions against current capabilities, training and exercises

• Coordinate training and exercise activities

• Identify and coordinate possible funding sources

30


Who? Whole Community Participants

31


How? Identify Priority Factors

Threats and Hazards• National threats and hazards• Jurisdictional threats and hazards• Hazard vulnerability analysis

Areas for Improvement/Capabilities

• Real-world incident corrective actions• Exercise corrective actions• Identified and/or perceived areas for improvement

External Sources Requirements

• Industry reports• State or national preparedness reports• Homeland security strategies

Accreditation Standards/Regulations

• Accreditation standards and/or requirements• Grants or funding-specific requirements• Occupational Safety and Health Administration

regulations

32


How? Risk AssessmentsThreat and Hazard Identification and Risk Assessment (THIRA) Guide (CPG 201)

Results used to define goals for Multi-year Training and Exercise Plan

33


How? Link Core Capabilities• Link each identified risk factor to the Core

Capabilities that mitigate the risk• Prioritize these Core Capabilities• Identify which stakeholder organizations provide

support toward mitigation of the risk

34


Mission Areas and Core Capabilities

35

Infrastructure Systems

Fatality Management Services

Access Control and Identity Verification

Mass Search and Rescue Operations

Logistics and Supply Chain Management

Mass Care Services

Public Health, Healthcare and Medical Services

Critical Transportation

Operational Communications

On-Scene Security, Protection and Law Enforcement

Situational Assessment

Health and Social Services

Environmental Response / Health and Safety

Screening, Search and Detection

Interdiction and Disruption

Intelligence and Information Sharing

Forensics and Attribution

PREVENTION PROTECTION RESPONSE RECOVERY

Housing

Community Resilience

Long-Term Vulnerability Reduction

Risk and Disaster Resilience Assessment

Threats and Hazard Identification

MITIGATION

Economic Recovery

Natural and Cultural Resources

Physical Protective Measures

Public Information and Warning

Operational Coordination

Planning

Risk Management for Protection Programs

and Activities

Cybersecurity

Supply Chain Integrity and Security

From the National Preparedness Goal (NPG), Updated: 2015Fire Management and

Suppression


Priorities Inform the TEP

36

Training &


Instructions:1. Read your assigned Jurisdictional Identity and complete the worksheet2. Have a group discussion of the THIRA process to determine training and

exercise needs and to identify and prioritize the Core Capabilities on which your jurisdiction intends to focus

3. Identify your jurisdiction’s top three Core Capabilities - since the groups are comprised of varied entities, you may need to compromise for the purposes of the activity

Objective:Use the THIRA process to determine your jurisdiction’s threats and hazards, your jurisdiction’s needs, and to identify and prioritize the Core Capabilities on which your jurisdiction intends to focus.

Activity 1: THIRA Process for Establishing Priorities

Time: 20 minutes with 20-minute report-back.

37


The State of

Columbia


Multi-year Training and Exercise PlanOutlines Program:• Characteristics

• Vulnerabilities/Challenges

• Priorities/Goals/Objectives

• Training and Exercise Methodology Targeted Core Capabilities

Planned Exercises

Training Courses

• Training and Exercise Plan (TEP)

39


Multi-year Training and Exercise Plan• Combination of exercises to accomplish program goals

and objectives

• Progressive approach exposes participants to increasingly complex exercises

• Training tied to exercises so training objectives are achieved or validated

40


Instructions:1. Determine existing training and exercises (Current Year)2. Determine future training and exercises required to meet Core Capabilities

(Year One, Year Two and Year Three)3. Write your training and exercises on the worksheet for Activity 24. Be creative – you will have to collectively determine your jurisdiction’s input

Objective:Building on Activity 1, prepare for a TEPW to coordinate training and exercise priorities, goals and types to support the Core Capabilities previously identified.

Activity 2: TEPW Schedule Development


41


Instructions:1. Place your sticky notes in the proper boxes according to your jurisdiction and

the appropriate timeframe for the current year (year one), year two and year three

2. Try to avoid planning efforts which may conflict with holidays, real-world events and other jurisdictional efforts (eg State of Columbia) which may require your support and participation.

3. Look to see where you may be able to combine efforts with another jurisdiction in order to maximize training and exercise efforts and save money

Objective:Building upon Activity 2, place your jurisdiction’s training and exercises on the State of Columbia calendar creating a State TEP

Activity 2.5: Create a TEP for the State of Columbia


42


Jurisdiction

Year 1Qtr 1 Qtr 2 Qtr 3 Qtr 4

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

State of Columbia

Capital City(Pine County)

Central City (Liberty County)

Clifton(Kane County)

StramfordCounty

GraniteCounty

WMD Awareness Level TNG

TTX – Mass Casualty Incident Plans

State TEPW

Regional TEPW

FSE Chemical Response

31 DEC

THIRA Due

HSEEPTNG

HSEEPTNG

Workshop Mass

Casualty Incident Plans

ICSTNG

DrillSEOC

15 JAN IEMC

Apps Due


Jurisdiction



State of Columbia




StramfordCounty

GraniteCounty

31 DEC

THIRA Due

State TEPW

Regional TEPW

Chemical Response

TNG

HSEEPTNG

HSEEPTNG

WKSP EvacPlans

TTX Hurricane

FE Flood

TTX COOP

COOPTNG

15 JAN IEMC

Apps Due

State IEMC


Jurisdiction



State of Columbia




StramfordCounty

GraniteCounty

31 DEC

THIRA Due

State TEPW

Regional TEPW

HSEEPTNG

HSEEPTNGWKSP

WMDTTX

WMDFSE

WMD

ICSTNG

ST & Nat ElectPol Conv

15 JAN IEMC

Apps Due


Rolling Summary of Outcomes• Analysis of exercise trends which guides the development

of future exercises

• This report is designed to: Inform elected and appointed officials on the progress of the

exercise program

Provide data to support preparedness assessments and reporting requirements

Enable exercise planners to modify objectives and the exercise schedule to reflect knowledge gathered from the exercises

• Not simply a collection of AARs from exercises

• Developed periodically throughout a series of exercises

46


Managing Exercise Program Resources• Budget Available financial resources (all stakeholders) Expected expenditures Monitoring and reporting requirements

• Staffing Administrative Operational

• Other Equipment, training, previous exercise materials,

MAAs, MOUs, MOAs, technical assistance, IT

47


Lesson 2: ReviewIn this lesson, we have discussed:• The purpose of a Training and Exercise Planning

Workshop (TEPW)• The purpose a multi-year Training and Exercise Plan

(TEP) and development and coordination of the multi-year Training and Exercise Schedule

• Exercise Program Manager’s role in managing program resources and how the Rolling Summary Report supports these efforts

48


Module 2: Exercise Program Management SummaryIn this module, we have discussed:• The importance of exercise program priorities to HSEEP

Program Management

• The purpose of The Training and Exercise Planning Workshop (TEPW) and its product: a multi-year Training and Exercise Plan (TEP)

• How to develop a progressive, comprehensive, integrated exercise program

• How to manage exercise program resources

49


Module 3:EXERCISE

DESIGN AND DEVELOPMENT


Module 3: Agenda

Lesson 1: Exercise Foundation

Lesson 2: Exercise Planning Teams

Lesson 3: Exercise Planning Activities

Lesson 4: Exercise Design

Lesson 5: Exercise Documentation

Lesson 6: Exercise Logistics


Lesson 1: Exercise FoundationEnabling ObjectiveAfter completing this lesson you should be able to describe how to use guidance from Program Management Planning in the design and develop of individual exercises.


Key Design and Development Steps• Reviewing elected and appointed officials’ guidance, Training and

Exercise Plan (TEP), and other factors

• Selecting exercise planning team and developing exercise planning timeline and milestones

• Developing exercise-specific objectives related to targeted core capabilities identified by elected and appointed officials

• Identifying evaluation requirements, identifying EEGs by mission area

• Developing the EEGs and exercise scenario

• Creating documentation

• Coordinating logistics

• Planning for exercise communication, control, and evaluation


Exercise Foundation Key Factors• Multi-year TEP• Jurisdiction’s existing plans and procedures• THIRA or other risk, threat, and hazard

assessments• Past exercise AAR/IPs• Grant or cooperative agreement requirements


Lesson 2: Exercise Planning TeamsEnabling ObjectivesAfter completing this lesson, you should be able to describe the:• Importance of ensuring representation of whole

community on planning team• Characteristics of a successful exercise planning team• Recommended management structure for exercise

planning teams• Role and function of the exercise planning team


Exercise Planning Team Representation• Manageable size aligned with exercise type or

scope/complexity• Represent full range of whole community

stakeholders and participating stakeholder organizations


Successful Planning Teams• Have clearly define roles, responsibilities, and functional

requirements• Engage senior and appointed officials and whole community

leadership in exercise planning• Utilize project management

principles• Follows standardized process• Can be organized using NIMS

Incident Command System (ICS) or otherstructure that defines support roles of each team member


Role and Function of the Planning TeamConducts planning meetings used to:• Determine exercise objectives,

evaluation plan, and control and simulation systems

• Design, develop, conduct, and evaluate results of exercise

• Develop scenario, EEGs and other exercise documentation

• Plan logistics for exercise conduct

• Identify, create, and distributes pre-exercise materials


Subject Matter Experts• Add expertise to the exercise planning team• Provide functional knowledge for player-specific

tasks evaluated through objectives• Help make the scenario realistic and plausible• Ensure appropriate evaluation of capabilities


Trusted Agents• Individuals on the exercise

planning team who may serve as facilitators, controllers or evaluators during the exercise

• DO NOT reveal scenario details to players prior to exercise conduct


Lesson 3: Planning ActivitiesFollowing completion of this lesson you should be prepared to identify and describe:• Exercise planning activities and how they

contribute to exercise development• Exercise development tasks associated with

each planning activity


Planning Activities (“Meetings”)• Meetings to discuss, review, or develop exercise

content• Forum for coordination and collaboration among

participating agencies and officials• Exercise planning teams should apply and adapt

HSEEP doctrine to meet their specific needs


Concept and Objectives Meeting (C&O)FOCUS: IDENTIFY THE SCOPE AND OBJECTIVES OF THE EXERCISE

DISCUSSION POINTS:• Propose exercise objectives and their aligned core capabilities

• Propose exercise location, date, and duration

• Participants and anticipated extent of play for exercise participants

• Exercise planning team

• Exercise control and evaluation concepts

• Exercise planning timeline and milestones


Concept and Objectives Meeting (cont.)TOOLS: Agenda Background briefing

OUTCOMES: Exercise concept Exercise timeframe Extent of participation Identification of planning team members Planning timeline, milestones, meeting dates


Initial Planning Meeting (IPM)FOCUS Utilize elected and appointed official guidance Exercise design requirements identified Develop exercise documentation Assigns roles and responsibilities

DISCUSSION POINTS Exercise objectives and core capabilities Evaluation requirements, including EEGs Relevant plans, policies, and procedures Exercise scenario Modeling and simulation planning Extent of play (time, date and location)


Initial Planning Meeting (IPM) (cont.)TOOLS: Read-ahead packet (Agenda, Core Capabilities, THIRA)

OUTCOMES:• Clearly defined exercise objectives and aligned core capabilities

• Initial capability targets and critical tasks

• Exercise scenario variables

• List of participating exercise organizations and anticipated organizational extent of play

• Identification of source documents

• Refined exercise planning timeline with milestones and lists of tasks Providing read-ahead materials (agenda, background information, purpose) result in more productive meetings.

Suggested Practice


Planning Meeting Follow-Up Activities Distribute IPM meeting notes Between meetings the planning team collaborates on

assignments and prepares draft exercise documentation Distribute draft documentation prior to next meeting Repeat activities for each follow-on planning meeting


Midterm Planning Meeting (MPM)PRIMARY FOCUS Re-engage elected and appointed officials Exercise organization Scenario and timeline development Logistics, and administrative requirements Review draft documentationDISCUSSION POINTS

Refinement and/or finalization of exercise documentation Logistical requirements Master Scenario Events List (MSEL) if no additional

MSEL planning meeting will be held


Midterm Planning Meeting (MPM) (cont.)

TOOLS: Read-ahead packetOUTCOMES: Reviewed or final exercise documentation (as applicable) Well-developed scenario to include injects (if no MSEL

Meeting held) Finalization of date, time, and location Identified logistics planning requirements Revised planning schedule, task assignments, meeting

date, and location for next meeting


SAMPLE MSELEvent No.

Time Title To From Controller Method Type Description/ Script

Expected Action Comments/ Notes

1

2

3

4

5

6


Master Scenario Events List (MSEL) MeetingPRIMARY FOCUS:Development of chronological list that supplements exercise scenario Event synopses, expected participant responses, objectives and core

capabilities Specific scenario events (or injects) that prompt players to implement the

plans, policies, procedures, and protocols that require testing during the exercise

Methods used to provide injects

DISCUSSION POINTS:• Tasks, conditions, and standards required to meet exercise objectives

• Key events and critical tasks

• Event originator, target player, expected player actions, and timeframe

• Contingency injects to prompt player action (if needed)


Master Scenario Events List (MSEL) Meeting (cont.)

TOOLS: MSEL Template/System Applicable plans, policies, and procedures

OUTCOMES: Key event injects and timeline for delivery Assignment of responsibility for constructing remaining

events Timeline for completion


SAMPLE MSELEvent No.

Time Title To From Controller Method Type Description/ Script Expected Action Comments/ Notes

1 0900 SimCell Operational

All Controllers

SimCell SimCell Phone Contextual Simulators are in position and communications are in place

Check communications with field controllers

2 0945 Communications check

All Controllers

SimCell All Radio Contextual Ensure communications are operable

All controllers establish communications with the SimCell

3 1000 STARTEX4 1001 Explosion Central City

Railroad Station

Pyrotechnics

Contextual Explosive detonated on site

N/A

5 1002 Report from Train Passenger

Central City Railroad Station Security

Passenger 1

SimCell Phone Contextual “There has been an explosion here in the railyard! A train car carrying barrels of some sort exploded and threw barrels all over the place!”

Call Central City Dispatch

6 1004 9-1-1 Call from Train Station

Central City Dispatch

Passenger 2

SimCell Phone Contingency

“I’m at the Central CityRailroad Station and there was an explosion. None of us can breathe. People are on the ground. Help!”

Dispatch, fire, EMS, and police to the Central City Railroad Station

To be injected if fire and EMS are not notified after initial call

7 1007 Police arrive on scene

N/A N/A Security N/A Expected Action

1st police cruiser expected on scene

Police officer should recognize signs of potential chemical contamination,position upwind/ uphill of site and don appropriate PPE


Final Planning Meeting (FPM)PRIMARY FOCUS: Re-engage elected and appointed officials—ensure

exercise aligned with intent, address questions, and receive last-minute guidance

Ensure all elements of exercise are ready for conduct

DISCUSSION POINTS: Final review of all exercise documentation Resolve any outstanding planning issues Review logistical activities (e.g., schedule, registration,

attire, special needs)


Final Planning Meeting (FPM) (cont.)TOOLS: Read-ahead packet

OUTCOMES: Final approval of exercise documentation and

presentation materials Identified issues resolved Attendees understand and approve exercise processes

and procedures Logistical elements and task assignments, including

equipment, facilities, and schedule confirmed


Lesson 4: Exercise DesignEnabling ObjectivesAfter completing this lesson, you should be able to describe the major components of exercise design including:

• Exercise scope

• Exercise objectives

• Evaluation requirements

• Exercise scenario


ScopeKey elements:• Type• Participation Level• Duration• Location• Parameters


Exercise Types

DISCUSSION-BASED

• Seminar• Workshop• Tabletop (TTX)• Game

OPERATIONS-BASED

• Drill• Functional Exercise (FE)• Full-Scale Exercise (FSE)


Discussion-based ExercisesTYPES• Seminars Plan Orientation/Review Assess Capabilities

• Workshops Analyze RequirementsDevelop Products

• Tabletop (TTX)Rehearsals Assess Plans

• GamesCompete/Collaborate Validate Plans Explore Consequences

FOCUS• Jurisdiction: Plans Policies Procedures

• Inter-Jurisdictional Agreements Understandings

PARTICIPANTS• Facilitators• Moderators• Evaluators• Stakeholders/Leaders/Players

79


Operations-based ExercisesTYPES• Drills Single agency/organization Provide training/skills reinforcement Validate procedures

• Functional Exercises Validate/Evaluate Capabilities Command-and-Control and

Coordination Functions

• Full-Scale Exercises Complex real-time responseMulti-agency cooperative – ICS Simulates reality – “as if” realMobilize and deploy resources and

personnel Prop and Actor involvement Requires close control and monitoring

FOCUS• Validate Implementation of: Jurisdictional

► Plans► Policies ► Procedures

Inter-Jurisdictional► Agreements ► Understandings

PARTICIPANTS• Controllers/Simulators• Evaluators• Actors• Players• Observers/VIPs

80


Blended Exercises

If you introduce actions and reactions into your discussion-based exercises, you wind up with blended exercises• Games• Seminars• Workshops• Tabletops


Exercise Participation LevelDefined by:• Organizations and levels of personnel required

to address identified objectives• Available resources and personnel of

participating organizations• Right size and duration to meet objectives• Compatible with venue location• Extent of Play Agreement (XPA)


Exercise Duration and Parameters• Time to address objectives effectively• Resource constraints and budget• Determines what to include in scenario• Define early to ensure manageable, realistic

exercise


What Are Objectives?• Driven by exercise program priorities• Cornerstone of scenario design, development,

exercise conduct, and evaluation• Follows SMART guidelines for development


Characteristics of Good Objectives

SMART Guidelines for Exercise Objectives

SpecificObjectives should address the five Ws: Who, What, When, Where, and Why. The objective specifies what needs to be done with a timeline for completion.

MeasurableObjectives should include numeric or descriptive measures that define quantity, quality, cost, etc… Their focus should be on observable actions and outcomes.

Achievable Objectives should be within the control, influence, and resources of exercise play and participant actions.

Relevant Objectives should be instrumental to the mission of the organization and link to its goals or strategic intent.

Time-Bound A specified and reasonable timeframe should be incorporated into all objectives.


Creation of Performance ObjectivesElement Tip Example

ActionStatement

Select observable action verb to describe what task(s) responders mustperform.

Incident Safety Officer (SO) conductsscene survey to identify hazard(s) and establish “Hot Zone” perimeter.

Condition by which the tasks must be performed

What is given/expected in executing the task? Skill, knowledge, tool(s), equipment, reference(s), chats(s)/scene survey, or standard operating procedures (SOPs) on which action should be based.

Using knowledge of HazMatclassification and OSHA’s Hazardous Waste Operations and Emergency Response (HAZWOPER) standard (29 Code of Federal Regulations [CFR] 1910.120)

PerformanceStatement

List what the player must be able to do/accomplish.

Determine hazard-specific health and safety risk to personnel in immediate range of incident.

CriteriaStatement

The performance standard used to measure achievement of objective.Described as:

• Degree of accuracy—how well?• Timeframe—when?• Speed/Distance—how much?

Within 30 minutes of arrival on scene.


Aligning Objectives to Core CapabilitiesCore

Capability

ActivitiesActivitiesCapability

TargetsCapability

Target

Observation Keys

• Task Criteria• Degree of Accuracy• Timeframe• Performance Standard

Critical Task TasksTasks

Critical Tasks

• Condition• Expected Action• Observation Key• Source: Determine Jurisdiction-Specific Action/SOP


Example of Core Capability AnalysisEnvironmental Response/Health and Safety

Capability Target:Hazardous materials release is rapidly identified and mitigated; victims exposed to the hazard are rescued, decontaminated, and treated; the impact of the release is limited; and responders and at-risk populations are effectively protected.


Selection of Critical TasksEnvironmental Response/Health and Safety

Critical Tasks: Coordinate rescue efforts with law enforcement to ensure safety of

rescuers while law enforcement secures incident site. Establish a hot zone (inner perimeter) to identify high hazard area(s)

where responders will operate. Monitor and control operating time of rescuers assigned to hot zone

to minimize rescuer exposure. Identify assets required for decontamination activities. Establish decontamination sites. Conduct decontamination. Decontaminate affected facilities and equipment. Conduct screening of affected persons.


STRATEGIC PLANS, POLICY-ORIENTED ISSUES

Example:Validate Central City’s existing response plan for (2) incident command (3) activation of essential HazMat personnel (1) during a chemical incident (4) to ensure alignment with NIMS and HazMat Response criteria.

1) Condition2) Who3) Action4) Standard

Objectives for Discussion-Based Exercises


RESPONSE SYSTEMS/TACTICAL-LEVEL:Example:During (1) a chemical incident, evaluate the ability of (2) Central City HazMat personnel , to (3) establish a Hot Zone and decontamination site (4) within 30 minutes of arrival on scene in accordance with existing SOPs.

1) Condition2) Who3) Action4) Standard – jurisdiction-specific

Objectives for Operations-Based Exercises


Central City’s Chemical FSE Objectives1. Central City’s incident command (who) shall provide essential HazMat

personnel to incident site (action) within 1 hour following notification (standard) of a chemical incident (condition) in accordance with existing SOPs (standard).

2. First responders (who) shall identify hazard and establish incident perimeter and hot zone (action) within 30 minutes of arrival (standard) on scene using established SOPs and appropriate hazard guidelines (standard) for the identified chemical (condition).

3. Emergency response personnel (who) shall administer proper levels of decontamination (action) to responders and victims prior to transport (condition) to hospitals in accordance with established procedures (standard).

4. Hospital staff (who) will evaluate HazMat victims upon arrival and determine decontamination status (action) of patients, and administer decontamination (action) as required prior to admittance (condition) to hospitals in accordance with established procedures (standard).


Objective: Develop two objectives that identify the specific actions/tasks, and measurement criteria or performance standard designed to demonstrate a core capability identified in your TEPW.

These objectives will be used in follow-on activities.

Activity 3: Develop Objectives

Time: 30 minutes, with 15-minute report back.

Instructions:1. Select two Core Capabilities from Activity 22. Determine the Critical Tasks you would like to see demonstrated during your

exercise which support those Core Capabilities3. Develop two SMART Objectives – one for each Core Capability selected


Evaluation Requirements• Developed early in exercise design process• Guide development of scenario, discussion

questions and/or MSEL• Identifies the capability targets and critical

tasks related to each core capability identified as an exercise program priority


The Exercise Scenario• Storyline for response activity or discussion• Three basic elements: Context Conditions Technical details

• Based on: Realistic Plausible threat Challenging

• Mechanism for assessing objectives and core capabilities


Developing an Exercise ScenarioA story of a simulated threat or hazard on which the exercise will focus• Based on organization’s threat/hazard risk assessment• Identifies Mission Area(s) targeted and provides exercise

context• May be supported by Modeling and Simulation

Model—representation of a system within point in time

Simulation—method of implementing performance model(s) over time


Activity 4: Developing an Exercise Scenario

Time: 15 minutes, with 10-minute report-back.

Objective: Develop a scenario of local significance, tied to the objectives developed in previous activity.

Instructions:1. Using the questions on the activity worksheet to guide your

discussion, develop a draft exercise scenario. Record your responses in the worksheet.

2. Identify and record jurisdiction-specific information, such as:• Incident location and time reported• Principle threat, hazard, or agent• Participating response agencies• Number of casualties


Lesson 5: Exercise DocumentationEnabling Objective:After completing this lesson, you should be able to identify exercise documentation requirements.


Exercise Documentation• Ensure accurate account of exercise• Control and distribution• Additional considerations


Exercise Documentation by Exercise TypeDocument Title Exercise Type Distribution Audience

Situation Manual (SitMan)Seminar (Optional), Workshop (Optional), TTX, Game

All Participants

Facilitator’s GuideSeminar (Optional), Workshop (Optional),TTX, Game

Facilitators

Multimedia PresentationSeminar (Optional), Workshop (Optional), TTX, Game

All Participants

Exercise Plan (ExPlan) Drill, FE, FSE Players, Observers

Controller and Evaluator (C/E) Handbook Drill, FE, FSE Controllers, Evaluators

Master Scenario Events List (MSEL)Drill, FE, FSE, Complex TTX (Optional), Game (Optional)

Controllers, Evaluators, Simulators

Extent of Play Agreement (XPA) FE, FSE Exercise Planning Team

Exercise Evaluation Guides (EEGs) TTX, Game, Drill, FE, FSE Evaluators

Participant Feedback Form All Exercises All Participants


Exercise Documentation (Discussion-based)• Situation Manual (SitMan)

Provided for discussion-based exercises

Provides the textual background and supports the scenario narrative and serves as primary reference material for all participants

• Facilitator Guide Helps facilitators manage discussion-based exercises

Outlines instructions and key issues for discussion

• Multimedia Presentation Illustrate scenario for participants

Intended to help focus and drive realism


Exercise Documentation (Operations-based)

• Exercise Plan (ExPlan) Given to players and observers

Provided for Operations-based Exercises to provide synopsis of planned exercise (scope, objectives and core capabilities)

Participant roles and responsibilities

Rules of Conduct and Safety and Security Policies

Communication methods

Maps of exercise site


Exercise Documentation (Operations-based) (cont.)

• Controller and Evaluator Handbook Describes the roles and responsibilities of exercise

controllers and evaluators and the procedures they should follow

Contains detailed scenario information

Logistics and communications plan

May be a separate document or an annex in the ExPlan

May be broken up into separate Controller and Evaluator versions


Master Scenario Events List (MSEL)• Chronological list of scripted events that drive exercise

play and specific functional area activity• MSEL events:

Contextual injects

Expected action events (milestones)

Contingency injects

• MSEL types Short: Inject, delivery time, short description, identifies responsible

controller, and recipient player

Long: Detailed description, exact quotes and formats for inject, and includes description of expected action


Elements of an Inject1. Number2. Expected Inject Time3. Responsible Controller4. Intended Player5. Event Synopsis6. Message/Description7. Expected Player Action8. Objective to be

Demonstrated9. Notes10.Injected By:

Number: Expected Inject Time:

Responsible Controller Intended Player:

Event Synopsis:

Message/Description:

Expected Player Action:

Objective to be Demonstrated:

Notes:

Injected By: __ Hard copy __ Telephone __ Face-to-face __ Audio tape __ Other (please specify)

__ Fax __ Radio __ E-mail __ Video tape

1 2

3 4

5

6

7

8

9

10


How to Develop a MSEL• Review capabilities

• Identify chronology of key actions

• Anticipate player actions

• Identify information resources

• Compile all MSEL events into single list

• Refine selected MSEL events; create detailed long version


MSEL Timeline Development


Mater Scenario Event List (MSEL)No. Time Title To From Controller Method Type Description/ Script Expected Player

Action Comments/

Notes

1 1000 STARTEX2 1001 Explosion Central

City Railroad Station

Pyrotechnics

N/A Explosive detonated on site

N/A

3 1002 Report from Train

Passenger

Central City

Railroad Station Security

Train PassengerJohn Doe

SimCell Phone Contextual “There has been an explosion here in the railyard! A train car carrying barrels of some sort exploded and threw barrels all over the place!”

Security should call Central CityDispatch to reportexplosion

Central City Railroad Station Controller needs to ensure that security calls dispatch – if not; notify Lead Controller

4 1005 9-1-1 Call from Train

Station

Central City

Dispatch

Train Station

AttendantJane Smith

SimCell Phone Contingency “I’m at the Central City Railroad Station and there was an explosion. None of us can breathe. People are on the ground. Help!”

Dispatch, fire, EMS, and police to the Salt Railroad Station

To be injected if fire and EMS are not notified after initial call

5 1010 Police arrive on scene

N/A N/A N/A N/A Expected Action

1st police cruiser expected on scene

Police officer should recognize signs of potential chemical contamination and position upwind/ uphill of site and don appropriate PPE


Activity 5: Developing MSEL and Injects


Objective: Understand what information the MSEL contains and practice developing injects

Instructions: 1. Familiarize yourself with format used for MSEL injects 2. Develop three injects: one contextual inject, one expected action inject and

one contingency inject for your exercise


Additional Exercise Documentation• Extent of Play Agreements (XPAs)• Exercise Evaluation Guides (EEGs)• Participant Feedback Form• Waiver Forms• Weapons Policy• Safety Policy


Media and Public Affairs Guidance• Inform public of community preparedness activities• Report on exercise and state of preparedness

following exercise • Press Releases Targeted to local media outlets (TV, radio, social

networks, newspapers)

Attendance/observation policy

• Public Announcement Inform and help to avoid confusion on the part of the

public


Media Policy• Media participation/notification determined by

sponsoring organization• Public Information Officer Escort media and ensure non-interference

• Separate real-world media from exercise media artificialities

• Inform public: Press releases Public announcements


Lesson 6: Exercise LogisticsEnabling ObjectivesAfter completing this lesson, you should be able to:

• Recognize logistical considerations critical to successful exercise conduct

• Identify facility considerations and venue specifics that influence development of scenarios for exercise conduct


Planning for Exercise LogisticsImportant but often overlooked and include consideration of:• Venues

Facility/Room

A/V

Supplies, food, and refreshments

• Badges and IDs Registration


Planning for Exercise Logistics (cont.)• Actors• Parking and transportation


Planning for Exercise ControlExercise Control maintains scope, pace, and integrity

• Staffing• Control structure


Planning for Exercise Control (cont.)• Controller Training• Communications Plan• Safety and Security


Module 3: Exercise Design and Development SummaryIn this module, we have discussed:

Exercise Foundation Exercise Planning Teams Exercise Planning Activities Exercise Design Exercise Documentation Exercise Logistics

In Module 4, you will learn: Exercise Conduct


Module 4EXERCISE CONDUCT

1


Module 4: AgendaThis module is broken into three lessons: Lesson 1: Exercise PreparationLesson 2: Exercise ConductLesson 3: Exercise Wrap-Up


Lesson 1: Exercise Preparation

Enabling Objectives• Describe activities related to exercise set-up• Describe the purpose of participant briefings• Identify participant roles and responsibilities

during exercise conduct


Discussion-based Exercise Set-UpPrior to Conduct:

Print exercise written materials

Arrange for presentation equipment and refreshments

Day of Conduct: Put up signage Set-up registration table(s) Set-up room and table

configurations Place name and table tents Test presentation equipment


Operations-Based Exercise Set-Up• Prior to Conduct:

Print exercise written materials Issue Media and Public Service Announcements Arrange briefing rooms and set-up A/V equipment Elected/Appointed Officials and Controller/Evaluator Briefings Walk-through exercise area and check for potential safety issues

• Day of Conduct: Place signage, set-up props, and mark

off exercise area Set-up registration area Actor, Player, Observer Briefings Conduct communications check


Exercise Orientation Briefings• Elected and Appointed Officials Briefing• Controller/Evaluator Briefing• Actor Briefing• Player Briefing• Observer Briefing


Participants• Exercise Director• Facilitators• Senior Controller• Controllers• Safety Controllers• Exercise Assembly

Area Controller

• Lead Evaluator• Evaluators • Simulators• Observers• Players• Actors


Lesson 2: Exercise Conduct Enabling ObjectiveAfter completing this lesson you should be prepared to describe the activities related to the conduct of discussion-based and operations-based exercises.


Discussion-based Conduct• Multi-media Presentation• Facilitated Discussion • Moderated Discussion• Exercise Data Collection


Operations-based Conduct• Realistic Representation of Capabilities• Actual or Simulated Environment• Functional Performance• Exercise Data Collection• Contingency Process


Control• Communication and Coordination• Roles and Responsibilities• Safe and Effective Play• Simulation


Lesson 3: Exercise Wrap-Up ActivitiesEnabling Objective After completing this lesson you should be able to describe the wrap-up activities that all relevant data is collected to support effective evaluation and improvement planning.


Wrap-Up Activities• Hot Washes• Data Collection• Debriefings


Instructions: Watch the video which outlines some tips and pitfalls of exercise design and development and take notes/answer questions on the worksheet.

Objective: Discuss the implications of exercise design and development choices.

Activity 6: Discussion-based Exercises Lessons-Learned

Time: 10 minutes, and a 10-minute report-out.


Instructions: Watch the video which outlines some tips and pitfalls of exercise design and development and take notes/answer questions on the worksheet.

Objective: Discuss the implications of exercise design and development choices.

Activity 7: Operations-based Exercises Lessons-Learned

Time: 8 minutes, and a 10-minute report-out.


Module 4: Conduct SummaryIn Module 4, we discussed:

Exercise Preparation

Exercise Conduct

Exercise Wrap-Up Activities

In Module 5, will describe the post-exercise evaluation process.


Module 5EXERCISE EVALUATION

1


Module 5: AgendaLesson 1: Evaluation Planning

Lesson 2: Exercise Observation and Data Collection

Lesson 3: Data Analysis

Lesson 4: Drafting the AAR

136


Lesson 1: Evaluation PlanningEnabling ObjectivesAfter completing this lesson you should be able to describe:• The composition of an evaluation team• Purpose and development of an Exercise Evaluation

Guides (EEGs)• The recruiting and assignment of evaluators• The documentation and tools used to conduct exercise

evaluations

137


Evaluation Planning PhaseInitial planning for evaluations includes:• Engaging senior elected and appointed officials

to identify specific evaluation requirements• Identifying evaluation requirements early in

planning and design phases• Ensuring consistency in evaluation method

138


Evaluation Team Responsibilities• Be familiar with the mission areas, core capabilities, plans,

policies and procedures to be examined during the exercise

• Determine the structure of the evaluation team

• Determine the tools and documentation needed to support the evaluation

• Conduct a pre-exercise C/E Briefing

• Recruit, train, and assign additional evaluators

139


Exercise Evaluation Team

140


Lead Evaluator• Oversees evaluation process and planning• Evaluation plan development using: Exercise-specific information

Plans, policies, and procedures

Determining evaluator assignments

Developing instructions and evaluation tools (e.g., EEGs)

141


Recruit, Assign and Train Evaluators• Recruited from non-participating organizations

• Identified early in planning process

• Assigned based on knowledge of functional area(s) and plans, policies, procedures, and agreements

• Trained on use of exercise documentation tools• Know what to look for and record during observation

142


Evaluation Documentation and Tools• Evaluation Plan

Exercise SitMan or Scenario C/E Handbook

• Evaluator Team Organization Assignments Locations C/E Directory

• Evaluation Instructions Instructions on use of tools, logs, forms

• Evaluation Tools Jurisdiction- or organization-specific SOPs EEGs MSEL

143


Exercise Evaluation Guides (EEGs)• Streamline and guide data collection

• Enable thorough assessment

• Support development of the AAR

• Provide a consistent process for assessing preparedness through exercises

• Help organizations map exercise results to exercise objectives, core capabilities, capability targets, and critical tasks for further analysis and assessment

144


Exercise Evaluation Guides (EEG) FormatDocument who, what, where, when and how tasks were completed and note artificialities related to each task.

145


Instructions: 1. Assign a member to be the recorder for your group2. Develop a single Core Capability EEG with customized Capability Targets

and Critical Tasks based on one of your jurisdiction’s objectives and cite at least one source for each Capability Target

3. Select one person to report back

Objective:Develop an Exercise Evaluation Guide (EEG)

Activity 8: Develop an EEG

Time: 20 minutes, with a 10-minute report-back.

146


Evaluator Time RequirementsEvaluators should be available for:

• Pre-exercise briefing and/or site visit

• Exercise conduct

• Post-exercise Hot Wash

• Controller and Evaluator Debriefing

• After-Action Meeting

147


Lesson 2: Exercise Observation & Data Collection

Enabling ObjectivesAfter completing this lesson, you should be able to describe:• The exercise observation data collection

process• The use of EEGs to record observations

148


Observation and Data Collection• Can differ between discussion-based exercises

and operations-based exercises• Forms the analytic basis for determining if critical

tasks were successfully demonstrated and capability targets were met

149


ObservationConducted to examine:• Utilization of plans, policies, and procedures related to

capabilities

• Implementation of legal authorities

• Understanding and assignment of roles and responsibilities of participating organizations and players

• Decision-making processes used

• Activation and implementation of processes and procedures

• How and what information is shared among participating agencies/organizations and the public

150


Data CollectionSupports AAR development by capturing:

• Decisions and recommendations

• Roles and responsibilities

• Coordination and cooperation

• Supplemental data/written records

Evaluators should not be a distraction or interfere with exercise play

151


Exercise ObservationsObservations come from a variety of sources, including:• Event logs• Video or audio recordings• Evaluator notes• Photographs• EEGs

152


Recording Observations• If and how quantitative or qualitative targets were met

• Actual time required for exercise players to complete the critical task(s)

• How target was orwas not met

• Decisions made andinformation gatheredto make decision

• Requests made andhow requests werehandled

• Resources utilized

• Plans, policies, procedures, or legislative authorities used or implemented

• Any other factors contributed to the outcomes.

153


Instructions: 1. Review EEG to become familiar with what to observe2. Watch video and record observations and time 3. Determine if required critical tasks were completed4. Comparing observations within your group5. Select a representative to report back your findings

Objective:Use the Exercise Evaluation Guide (EEG) activities and tasks to capture exercise observations.

Activity 9: Recording Observations


154


Lesson 3: Data AnalysisEnabling ObjectiveAfter completing this lesson, you should be able to describe the processed used to analyze data to determine root causes for capability shortfalls.

155


Data Analysis Process• Consolidation of data• Examine and compare performance against

targets• Identify strengths and areas for improvement• Conduct root-cause analysis• Inform stakeholders of underlying causes within

shortfalls

156


Root-Cause AnalysisWere the capability targets met?

If not, WHY not?WHAT factors contributed?

157


Lesson 3: ReviewIn this lesson we have described:The process used to analyze data to determine root causes for capability shortfalls.

158


Lesson 4: After Action Report (AAR) Draft

Enabling ObjectivesAfter completing this lesson you should be prepared to discuss the:• Purpose of the After-Action Report (AAR)• Elements of an AAR• AAR review process

159


What Is an AAR?• Summarize key information related to evaluation• Overview of performance related to each

exercise objective and associated core capabilities

• Length, format, and development timeframe of the AAR depend on the exercise type and scope

160


Elements of an AAR• Exercise Overview• Analysis of Core Capabilities• Appropriate Appendices (e.g., Improvement

Plan, Participant List)

161


Activity 10: Writing AAR Contribution


162

Instructions:1. Review Recorded Observations from Activity 92. Using areas noted for improvement create a single AAR contribution3. Enter this on the worksheet4. Be prepared to share with group during the report back

Objective:Develop the highest priority AAR contribution to improve jurisdictional preparedness.


AAR ReviewExercise sponsor distributes AAR Draft to participating organizations and elected and appointed officials who:• Review and determine areas for improvement• Determine organization with responsibility for

corrective actions

163


Module 5: Evaluation SummaryIn this module, we have discussed the first four steps of the evaluation and improvement process:• Plan and organize the evaluation• Observe the exercise and collect data• Analyze exercise data• Develop the Draft AAR

This information leads us to Module 6—Improvement Planning.

Module 6: Improvement

Planning

164


Module 6IMPROVEMENT PLANNING

165


Identifying Corrective ActionsEnabling ObjectivesAfter completing this lesson, you should be able to describe the processes used to identify, prioritize, and implement corrective actions, including:• The After Action Meeting• After-Action Report (AAR)/Improvement Plan (IP)

finalization• Corrective Action tracking and implementation• Using improvement planning to support continuous

improvement

166


Corrective ActionsConcrete, actionable steps to resolve shortfallsCorrective actions may include:• Changes to plans and procedures,

organizational structures, and/or management processes

• Additional training, equipment, or resources

After review and confirmation of areas for improvement a Draft Improvement Plan is developed.

167


Corrective Actions• Cover sustainment and improvement issues• State what should be done and who should do it• Use a positive, active voice• Link to observations and analysis• Be consistent with other recommendations

168


Sample Corrective Actions• The chief of plans should attend the EMI course

on developing an Incident Action Plan.• The nine counties should develop a regional

CBRNE task force. • The city and county should sustain the Unified

Command that integrates their response to a disaster.

169


Better Corrective Actions• The chief of plans should attend the EMI course on

developing an Incident Action Plan during the next fiscal year as required by his job description.

• The nine counties should develop a regional CBRNE task force within six months as per State directives and Department of Homeland Security grant requirements.

• The city and county should sustain the Unified Command that integrates their response to a disaster, and they should practice establishing UC and other ICS principles during annual exercises.

170


Objective: Write two Corrective Actions that are focused and applicable to the State of Columbia’s preparedness.

Activity 11: Develop Corrective Actions


Instructions:1. Based on the gaps and issues recorded in the previous activity, work

with your table group to develop two Corrective Actions2. Record these recommendations on the Writing Corrective Actions

Worksheet3. Select two people to represent the group – one to read each of the

recommendations in the report-back session


After Action Meeting (AAM)• Forum to review the revised AAR and the Draft IP• Final consensus on draft corrective actions• Develop deadlines for implementation of corrective

actions• Identify specific corrective action owners and

assignees

172


Finalizing the AAR/IP—Corrective Action Tracking and Implementation• Distributed to exercise planners, participants,

and other preparedness stakeholders as appropriate

• Tracking corrective actions to completion• Ensure a system is in place to validate previous

corrective actions have been successfully implemented

173


Using IPs to Support Continuous Improvement

The Principles of Continuous Improvement:• Consistent Approach• Support National Preparedness• Effective Issue Resolution and Information Sharing• Application across Operational Phases

174


Why Exercise?Answer:Conducting exercises and documenting the strengths, areas for improvement, and associated corrective actions is an important part of the National Preparedness System, and contributes to the strengthening of preparedness across the Whole Community and achievement of the National Preparedness Goal. Over time, exercises should yield observable improvements in preparedness for future exercises and real-world events.

175


Introduction and Fundamentals• HSEEP is a consistent approach to capabilities-based

exercise program management that uses a common methodology to measure progress toward building, sustaining, and delivering core capabilities.

• HSEEP Fundamental Principles Guided by Elected and Appointed Officials

Capability-Based and Objective-Driven

Utilizes Progressive Planning Approach

Whole Community Integration

Informed by Risk

Common Methodology

176


Program Management• Elected and Appointed Officials• Training and Exercise Planning Workshops (TEPW) Whole Community

THIRA

Core Capabilities

• Training and Exercise Plans (TEP) Goals and Objectives

Progressive Approach

Rolling Summary of Outcomes

177


Design and Development• Exercise Foundation

• Exercise Planning Teams

• Exercise Planning Activities

• Exercise Design

• Exercise Development—Documentation

• Exercise Development—Logistics

178


Exercise Conduct• Exercise Preparation• Exercise Conduct• Exercise Wrap-Up

• Evaluation Planning• Exercise Observation and Data Collection• Data Analysis• Drafting the AAR

179

Exercise Evaluation


Improvement Planning• The After Action Meeting

• After-Action Report (AAR)/Improvement Plan

(IP) finalization

• Corrective Action tracking and implementation

• Using improvement planning to support

continuous improvement

180

Documents

Homeland Security Exercise Evaluation Programwyohomelandsecurity.state.wy.us/training/L0146 HSEEP...Module 1: Introduction and HSEEP Fundamentals L0146 Version 13.1 HSEEP Training