INDEPENDENT VERIFICATION AND VALIDATION (IV&V) OF REAL ...rtepm.vmasc.odu.edu/Appendix_D_IVV.pdf · The Commonwealth of Virginia and the Virginia Department of Emergency Management

i

INDEPENDENT VERIFICATION AND VALIDATION (IV&V)

OF

REAL TIME EVACUATION PLANNING MODEL (RtePM)

Work performed for

Virginia Modeling and Simulation Center (VMASC)

Old Dominion University

by:

DDL Omni Engineering

440 Viking Drive Suite 200

Virginia Beach, VA 23452

22 April, 2013

Mr. Eugene Nielsen Mr. Thomas Cole

Project Lead Vice President, Simulation and

Systems Division

ii

EXECUTIVE SUMMARY

This independent assessment of the Real Time Evacuation Planning Model (RtePM) was

conducted by DDL Omni Engineering to verify and validate the capabilities of the RtePM. The

tasks were performed in accordance with the following Statement of Work:

Regional Catastrophic Preparedness Grant Program: Real-Time Evacuation Modeling and

Simulation for the National Capital Region

RtePM provides evacuation planners with a tool that allows multiple scenarios to be developed,

saved, and evaluated for evacuations from any type of hazard from any location in the

continental United States. RtePM was shown to be applicable to a wide variety of scenarios

specific to a geographic location.

Normally, verification is conducted prior to or during model development. Since the RtePM

model had already been developed, verification was not accomplished in the normal sense.

Instead DDL Omni Engineering took the approach to identify what an evacuation

model/simulation should have in order to be an effective evacuation tool. We called this

conceptual model validation and identified eight critical factors a model/simulation must have in

order to be an effective evacuation model/simulation. These critical factors and their associated

essential elements are in Appendix B.

We determine that RtePM met six of the eight critical factors. Weather and emergency services

were critical factors that RtePM did not demonstrate the ability to handle. However, we believe

workarounds such as changing speed limits, or closing roads/bridges/tunnels can simulate the

adverse effects of weather.

We also validated RtePM by using the six validation techniques described in “Verification and

Validation of Simulation Models” by Robert G. Sargent. These validation techniques are: 1)Data

Validity; 2) Comparison to other models; 3) Event Validity; 4) Internal Validity; 5)Parameter

Variability-Sensitivity Analysis; and 6) Historical Validation.

The first validation technique we used verified that the data residing in RtePM was accurate and

useful in the determination of evacuation time. We specifically looked at population and road

accuracy. Results indicated population data in RtePM was consistent with the U.S. Census 2010

and road data was accurate to acceptable levels. Model validity is highly dependent on input

data. DDL Omni Engineering recommends that reviews of population and road data occur on a

regular basis to maintain infrastructure currency. Users should take advantage of the

customization features of RtePM that allow for those with local knowledge to make corrections,

updates, and seasonal variations as indicated.

iii

The second validation technique we considered was a comparison of RtePM evacuation times to

other evacuation model times. Evacuation models considered include HURREVAC, OREMS,

CUBE and VISSIM. Because of the differences between models, the issue of comparing models

to models and the limited time frame for learning new software, the model to model comparison

was not performed.

The third validation technique, Event Validity, compared Nuclear Regulatory Commission

(NRC) evacuation time estimates to RtePM evacuation times and compared RtePM to post-

hurricane assessments. Assessments from Hurricanes Hugo, Opal, Floyd, Andrew, Ike, and

Bonnie were used. Actual clearance times obtained from these assessments matched RtePM

model times 95% of the time.

Model reliability was tested with the Internal Validity validation technique. When the user

simulates an evacuation several times in a deterministic model without any changes in the

scenario, it would be reasonable for the user to expect the evacuation times would remain the

same. RtePM reliability was validated.

The next validation technique tested was the Parameter Variability-Sensitivity Analysis

technique. It consisted of changing RtePM input and initial condition parameters to determine

the effect upon the model and its output. By running RtePM scenarios several times and

changing inputs to the scenario, we were able to determine that results produced by RtePM are

dependable and stable.

The last validation technique was to validate the Johns Hopkins University/Applied Physics

Laboratory (JHU/APL) Comparison Document, dated May 2012. This document compared

the output of RtePM to actual evacuation numbers from select real world evacuations.

JHU/APL also conducted a Face Validation that included positive comments about RtePM

from Federal, State and Local emergency managers. When comparing the data from

JHU/APL, US Army Corp of Engineers (USACE), and DDL Omni Engineering results were

consistent.

As a result of this analysis, DDL Omni Engineering determined that the current version of

RtePM is a fast, reliable, and easy to use system. It should be made accessible to evacuation

planners at all levels of government to effectively and efficiently plan evacuation routes for any

type of disaster. Federal, State and local emergency managers must have accurate, reliable

evacuation plans when the need arises, without spending months to develop a new study that

may cost thousands of dollars. RtePM is easy to use, and the effectiveness of the system

capabilities can be realized even by the first time user.

iv

Contents EXECUTIVE SUMMARY ................................................................................................................................... i

PURPOSE ....................................................................................................................................................... 6

BACKGROUND ............................................................................................................................................... 6

VERIFICATION and VALIDATION TECHNIQUES .............................................................................................. 8

CONCEPTUAL VALIDATION ........................................................................................................................... 8

OPERATIONAL VALIDATION .......................................................................................................................... 9

OPERATIONAL VALIDATION RESULTS ......................................................................................................... 10

1. Data Verification/Validity ................................................................................................................... 10

Table 1 Population Comparisons ........................................................................................................ 11

Figure 1 Roadway Comparison A ........................................................................................................ 13

Figure 2 Roadway Comparison B ........................................................................................................ 14

Figure 3 Roadway Comparison C ........................................................................................................ 14

Figure 4 Roadway Comparison D ........................................................................................................ 15

2. Comparison to other Models ............................................................................................................. 15

Table 2 Comparison Models ............................................................................................................... 16

3. (A) Event Validity - Nuclear Regulatory Commission Evacuation Studies ........................................... 16

Figure 5 North Anna NRC Comparison ................................................................................................ 18

Figure 6 Bellefonte NRC Comparison .................................................................................................. 19

Figure 7 Callaway NRC Comparison .................................................................................................... 20

Figure 8 Nine Mile Point NRC Comparison ......................................................................................... 21

3. (B) Event Validity - Comparison to Real World Assessments ............................................................ 21

Table 3 Real World Comparison.......................................................................................................... 22

4. Internal Validity Stability - Reliability ........................................................................................... 25

Table 4 Reliability Confidence ............................................................................................................. 26

5. Parameter Variability ......................................................................................................................... 26

Table 5 Parameter Variability ............................................................................................................. 27

Table 6 Change in Population and Seasonal Scalability Comparisons ................................................ 29

Table 7 Traffic Density (PPV and Background Traffic) Comparisons ................................................... 30

Table 8 Road Closures and Contra Flow Comparisons........................................................................ 31

6. Historical Data Validation .................................................................................................................. 32

v

Figure 9 Hancock County Comparison ................................................................................................ 33

Figure 10 Jackson County Comparison Scenario A ............................................................................. 34

Figure 11 Jackson County Comparison Scenario B ............................................................................. 35

Table 9 Ocean City Express Way (RT90) Vehicle Count ...................................................................... 37

Figure 12 Ocean City Express Way (RT90) Graph ............................................................................... 37

Table 10 Coastal Highway (RT1) Vehicle Count .................................................................................. 38

Figure 13 Coastal Highway (RT1) Graph.............................................................................................. 39

Table 11 Ocean Gateway (RT50) Vehicle Count ................................................................................. 39

Figure 14 Ocean Gateway (RT50) Graph ............................................................................................. 40

Table 12 Background Traffic ............................................................................................................... 40

Table 13 End Point Weighting ............................................................................................................. 41

DOCUMENT SUMMARY .............................................................................................................................. 43

CONCEPTUAL VALIDATION ................................................................................................................. 43

OPERATIONAL VALIDATION ................................................................................................................ 44

RECOMMENDATIONS ................................................................................................................................. 47

REFERENCES ................................................................................................................................................ 48

APPENDIX A ................................................................................................................................................. 51

TEST PLAN ........................................................................................................................................... 51

APPENDIX B ................................................................................................................................................. 66

CRITICAL FEATURES AND ESSENTIAL ELEMENTS ................................................................................ 66

APPENDIX C ................................................................................................................................................. 75

RtePM ENHANCEMENTS ..................................................................................................................... 75

6

PURPOSE

The purpose of this report is to provide results of the Independent Verification and Validation

(IV&V) of the Real Time Evacuation Planning Model. DDL Omni Engineering conducted the

study to validate the capabilities and limitations of RtePM for the Virginia Modeling, Analysis

and Simulation Center (VMASC). The assessment also provides the identification of additional

capabilities and potential improvements as required by the Statement of Work. Focus was

centered on ensuring RtePM meets system requirements and fulfills its intended purpose.

BACKGROUND

RtePM was developed in response to the emergency management community’s need for a tool to

provide quick estimates of the time required to evacuate an area in response to a natural or man-

made disaster.

The U.S. Department of Homeland Security, Science and Technology (DHS S&T) tasked John

Hopkins University’s Applied Physics Laboratory (JHU/APL) with the research, development

and delivery of an evacuation tool prototype that will enhance the capability of state and local

emergency managers to plan for and execute emergency evacuations.

RtePM Purpose

Provide a planning tool to assist end user planning for all hazard events by utilizing a

color-coded map interface. RtePM uses highway network data and census data to enable

the user to work through an evacuation scenario, setting parameters to model specific

conditions and responses. After the scenario has simulated the traffic flow and

calculated clearance times, an animated graphical depiction can be displayed.

RtePM Users

RtePM is for use by emergency management personnel for evacuation planning.

RtePM Output

RtePM provides evacuation times for user defined geographic regions and identifies

potential impediments to a timely evacuation. The tool allows the end user to quickly

try different “what-if" scenarios and to rapidly modify existing evacuation plans.

Initial planning called for two phases with the first phase using static data injection for planning

evacuations and the second phase expanding the static mode and adding a dynamic mode to

enhance capability and flexibility. 1

1 RtePM Statement of Work, August 2012

7

The Commonwealth of Virginia and the Virginia Department of Emergency Management

(VDEM) are coordinating on a project funded by the Regional Catastrophic Preparedness Grant

Program: Real-Time Evacuation Modeling and Simulation for the National Capital Region

(NCR). Specifically, the project employs modeling and simulation techniques to advance the

planning for and execution of a safe evacuation of residents in the NCR and the surrounding

states following an attack by terrorists, such as the release of toxic gas or detonation of a “dirty

bomb”.2

DDL Omni Engineering was tasked to provide an IV&V of RtePM. Confidence in RtePM must

be gained before its results are used to make decisions involving large sums of money or risk to

human life. Model verification and validation are critical in the development of a

model/simulation. Complicating this effort is the fact that no set of specific tests can easily be

applied to determine the “correctness” of the model. Furthermore, when IV&V is conducted

after the simulation model has been completely developed, as in the case of RtePM, the

evaluation performed can range from simply evaluating the validation conducted by the model

development team to performing a complete verification and validation effort.3

2 RtePM Statement of Work, August 2012 3 VERIFICATION AND VALIDATION OF SIMULATION MODELS Robert G. Sargent, Proceedings of the 2010 Winter Simulation Conference

8

VERIFICATION and VALIDATION TECHNIQUES

Verification and Validation (V&V) assessment activities primarily deal with the measurement

and assessment of accuracy of models and simulations (M&S). Verification is concerned with

the notion of correctly constructing the model, i.e., building the model right. It deals with

converting a problem formulation into a model specification or into a characterization accurately.

Validation refers to the notion that the right model was built and behaves with acceptable levels

of accuracy that are consistent with the goals and objectives of the model.

There are two distinct types of validation: conceptual and operational validation. Conceptual

model validation (validation of design documents) is the determination (usually by a group of

subject matter experts (SME) that the assumptions underlying the proposed conceptual model are

correct and that the proposed model/simulation design elements and structure (i.e., the

model's/simulation’s fidelity) likely will lead to results realistic enough to meet the requirements

of the application. Operational validation (hands-on the model/simulation) compares the

responses of the model/simulation with known or expected behavior from the subject it

represents to ascertain that those responses are sufficiently accurate for the range of intended

uses of the model/simulation.

CONCEPTUAL VALIDATION

Normally, verification is conducted prior to or during model development. Since the RtePM

model had already been developed, verification was not accomplished. Instead DDL Omni

Engineering took the approach to identify what an evacuation model/simulation should have in

order to be an effective evacuation tool. We called this conceptual validation and the following

steps were accomplished in order to determine whether RtePM had the features of an effective

evacuation tool:

1. Identified the purpose of RtePM and the intended users of the model/simulation. This

was accomplished in the purpose and background section above.

2. Defined critical features that an evacuation planning model must have in order to meet

the purpose of the planning model. DDL Omni Engineering identified the eight critical

features below.

a) Type of disaster- A good model will determine evacuation time for any type of

disaster.

b) Population density- The model must accurately model populations.

c) Roadway configuration- The model must accurately display roads and routes.

d) Traffic Density- The model must consider the effects of congestion.

e) Time to Evacuate- The model must accurately determine evacuation time.

9

f) Emergency Services- The model should model the effects of emergency services.

g) Capabilities- The model/system should have user friendly graphical user

interfaces (GUIs) and easy to use displays.

h) Weather- The model/system should consider the impact of weather.

3. Developed essential elements or requirements of a planning model to make the critical

features function properly.

4. Developed validation use-cases that will test all essential elements. See Appendix A.

5. Performed validation, using the use-cases, to determine the model’s/simulation’s ability

to meet the essential elements.

6. Determined that RtePM met all the critical features that an effective evacuation planning

model/simulation should have except considering the impact of weather and the effects of

emergency services. However, we believe workarounds such as speed limits,

road/bridge/tunnel closures can simulate the effects of weather. The results of RtePM’s

ability to meet the critical features and essential elements are documented in Appendix B.

OPERATIONAL VALIDATION

To accomplish operational validation, we used Robert Sargent’s article listing the following

techniques to develop model/simulation confidence. Each of these techniques has specific

application criteria that were used as a guideline for model validation. We also discussed the

strengths and limitations of each technique particularly as it applies to validation of RtePM.4

Data Validity- Ensuring that the data necessary for model building, model evaluation and

testing, and conducting the model experiments to solve the problem are adequate and correct.

This is an excellent technique for verifying databases and we used it to validate the

population and roadway facilities in RtePM. The limitation here is databases can be very

large making it nearly impossible to validate the entire database.

Comparison to other models- Various results of the simulation model being validated are

compared to results of other (valid) models. The simulation model is compared to other

simulation models that have been validated. At first this technique appears to be a good

method to check the model’s/simulation’s ability to stand up to other models/simulations.

But this technique has several limitations including: 1) The models/simulations must be

4 VERIFICATION AND VALIDATION OF SIMULATION MODELS

Robert G. Sargent, Proceedings of the 2010 Winter Simulation Conference

10

similar, e.g. both should be evacuation models. 2) The models/simulation must use similar

databases, i.e. you not only do not want to compare apples to oranges but you may not want

to compare Red Delicious apples with Granny Smith apples. 3) The models/simulations

being compare to must be validated or the model/simulation might be very wrong. 4)

Models/simulations must have similar functionality, i.e. how you get at the data and how you

manipulate the data must have some commonality in order to do a meaningful comparison.

Event Validity- The “events” of occurrences of the simulation model are compared to those

of actual events to determine if they are similar. This is comparing the results of running your

model/simulation to real world results, e.g. model/simulation evacuation times are compared

to actual evacuations for hurricanes or the NRC evacuation case studies.

Internal Validity- Several runs of a model are made to determine the amount of (internal)

variability in the model. A large amount of variability (lack of consistency) may cause the

model’s results to be questionable. This is fairly straight forward, make several runs and

check for consistency. The problem comes when there are inconsistencies, determining what

caused them and why.

Parameter Variability-Sensitivity Analysis- This validation technique consists of changing a

model’s input and initial condition parameters to determine the effect upon the model and its

output. A different output is expected if the input or initial conditions are changed.

Sometimes the output does not change because the model/simulation compensates for

changes so you must be aware of how the model/simulation responds to a changed input.

Historical Validation

Historical validation evaluates the verification and validation efforts that have already been

performed by other agencies including the original developer. Historical validation is useful

when IV&V is conducted after the model/simulation has been completely developed. When

historical validation data is available, the data can be used for testing other models/

simulations to determine their validity. Historical validation may include several validation

techniques such as comparing model results, parameter variability, event validation and face

validation. Face validation determines if the model seems reasonable to people who are

knowledgeable about evacuation planning. Face validation is based on the look and feel of

the model and the results. The limitation of this validation technique is that it can become

subjective. It may also be difficult to obtain qualified SMEs.

OPERATIONAL VALIDATION RESULTS

1. Data Verification/Validity The first operational validation technique we used was to verify that data residing in RtePM

11

was accurate and useful in the determination of evacuation time. This was accomplished by

comparing two of the eight critical factors we identified that an evacuation model/simulation

should have against actual data sets within RtePM. We evaluated population and roadway

configuration.

The first data set verified was population. RtePM utilizes the Oak Ridge National Laboratory

(ORNL), Department of Computational Sciences and Engineering Division population

database known as LandScan™. This database relies heavily on the 2010 U.S. Census data.

LandScan is the standard for global population distribution, and is accepted for estimating at

risk population by the Department of Defense (DOD) and Department of State (DOS).

Table 1 shows randomly chosen population data for five census block groups for seven

representative cities throughout the nation. The first column in the table lists census block

identification areas in the highlighted city. The second column lists the population for census

block in that row that we found in U.S. Census/ LandScan data. The third column lists the

population for the census block in that row that we found in RtePM data.

The population data for RtePM is identical to the U.S. Census/LandScan data in all cases,

verifying that RtePM population data is accurate.

Table 1 Population Comparisons

WASHINGTON DC

Census Block ID US Census/Landscan RtePM

110010108001 599 599

110010055004 1097 1097

110010043001 2062 2062

110010053014 1551 1551

110010108003 2008 2008

VIRGINIA BEACH, VIRGINIA

Census Block ID US Census RtePM

518100442002 2511 2511

518100444011 648 648

518100448061 2081 2081

518100440013 1076 1076

518100440034 874 874

JACKSONVILLE, FLORIDA


120310003002 725 725

12

120310029022 1402 1402

120310174001 1345 1345

120310002002 1005 1005

120310147012 3876 3876

MOBILE, ALABAMA


10970004021 103 103

10970011001 668 688

10970009031 1627 1627

10970004012 262 262

10970005001 1403 1403

NEW ORLEANS, LOUISIANA


220510232002 1911 1911

220510214001 1176 1176

220510239002 784 784

220510201022 942 942

220510202012 1288 1288

SAN DIEGO, CALIFORNIA


60730039013 1636 1636

60730039021 1088 1088

60730111001 1851 1851

60730051002 4218 4218

60730050001 2221 2221

SEATTLE, WASHINGTON


530330090002 1520 1520

530330089003 1190 1190

530330088003 944 944

530330086002 2503 2503

530330091001 1243 1243

The second data set verified was roadway configuration. We wanted to determine that the

model utilized the most current and accurate roads and routes. The technique we used to do

this was to compare RtePM’s HSIP-Gold 2010 NAVTEQ highway network data, to known

roadway facilities in the Hampton Roads area. We selected fifteen well known roads and

routes in the Hampton Roads area to evaluate and found all fifteen to be accurately portrayed

in RtePM.

13

We also compared roadway facilities in thirty areas outside the Hampton Roads area by using

Google Maps, which map the earth by the superimposition of images obtained from satellite

imagery, aerial photography and GIS 3D globe. We selected thirty areas where evacuations

are likely to occur and compared Google Maps of the selected roadway to RtePM maps and

data of that roadway. In twenty nine of thirty comparisons, the roadway facilities accurately

compared as indicated in Figure 1 and Figure 2. These Figures, for Gulf Shores Parkway (AL

59), showed a four lane divided highway in both the Google Maps and RtePM maps and data.

This verified that the RtePM data was correct.

In one of the thirty comparisons, the RtePM data was not consistent with Google Maps of the

area or the RtePM map of the area. Figures 3 (maps and data on the coastal highway at the

Maryland/Delaware state line) showed the RtePM data (outlined in red) was inaccurate listing

only one lane of traffic on this corridor, although the RtePM picture clearly shows two lanes

of traffic each way. The Google Map picture in Figure 4 also shows two lanes of traffic each

way, demonstrating the need for locally knowledgeable users to make inputs for accuracy and

currency.

Figure 1 Roadway Comparison A

14

Figure 2 Roadway Comparison B

Figure 3 Roadway Comparison C

15

Figure 4 Roadway Comparison D

SUMMARY: Using the Data Validity technique of operational validation, DDL Omni

Engineering found that the critical feature “population” was modeled accurately in RtePM

data. RtePM population data was identical to U.S. Census/ LandScan population data for the

seven U.S. cities and thirty-five census id blocks sampled. Users can be confident that RtePM

population data is accurate.

The critical feature “roadway configuration” that every model/simulation should have in order

to accurately simulate evacuations was evaluated by DDL Omni Engineering using the Data

Validity technique. In twenty-nine of the thirty instances that we tested for road data

accuracy, RtePM maps and data accurately compared to Google Maps and known roadway

facilities in the Hampton Roads area. In one of the thirty instances tested, the road model data

was inconsistent with the RtePM map and Google Maps and may affect evacuation time

results.

DDL Omni Engineering recommends the RtePM ‘Roads Tab’ be updated by local emergency

managers who are familiar with local roads and routes.

2. Comparison to other Models DDL Omni Engineering attempted to compare RtePM evacuation times to evacuation times the

models listed in Table 2 generated. Because of the differences between the models, the issues of

comparing models to models listed in the operational validation lead-in above and the limited

timeframe for learning to operate each new model, the model to model comparison was not

performed.

16

Table 2 Comparison Models

MODEL PURPOSE

HURREVAC (HURRicane EVACuation)

Storm tracking and decision support tool.

Provides evacuation decision time to

emergency managers

OREMS (Oak Ridge Evacuation Modeling

System)

A microcomputer-based system for

simulation of traffic flow during an

emergency evacuation.

CUBE Voyager – macroscopic modeling

simulation

Provides a comprehensive library of

functions for the modeling and analysis of

passenger transport systems: roadways,

public transit, pedestrians and bicycles

CUBE Avenue

An extension of CUBE voyager, mesoscopic

modeling simulation which models traffic at

greater detail than Cube Voyager

VISSIM - Verkehr In Städten -

SIMulationsmodell” (German for “Traffic in

cities - simulation model)

Microscopic traffic flow simulation model

SUMMARY: Comparison of other evacuation models with RtePM was not successful

because learning the operation of compatible models was difficult due to time constraints,

complexity of other models, and the lack of comparable capabilities for those models to those

of RtePM. The resulting validation would be suspect, with a reduced level of confidence in the

accuracy of the comparisons. Comparison of the RtePM model to other models was not

accomplished.

3. (A) Event Validity - Nuclear Regulatory Commission Evacuation Studies

The NRC requires estimated evacuation times in the event of a major accident at a commercial

nuclear power station. The exposure of the public to airborne radioactive materials can be

prevented or greatly reduced by evacuating the area immediately surrounding the reactor site.

Reactor licensees are required to conduct studies to estimate the time needed to evacuate the

public from the area surrounding each nuclear power station. The results of such studies are used

17

by regulatory personnel and emergency planners to assess the potential effectiveness of

protective responses for the public. The time required to evacuate the public from a 2, 5, or 10-

mile radius is estimated by analyzing the available transportation facilities and other relevant

conditions within this radius.

Four NRC evacuation time studies were compared to evacuation estimates modeled by RtePM.

Figures 5, 6, 7 and 8 display the results of the comparison. The first three studies (North Anna,

Bellefonte, and Callaway) simulate summer, midweek, mid-day, and good weather. The final

study (Nine Mile Point) simulates winter, weekend, mid-day with snow. The results display NRC

and RtePM evacuation times for a 2 mile radius, 5mile radius, and a 10 mile radius. The

comparison results are as follow:

1. Dominion Power, North Anna 3 Nuclear Power Plant Emergency Plan, Dec 2008

This plan describes the analyses undertaken by a study to develop an evacuation time estimate

(ETE) for the North Anna Power Station (NAPS) in Virginia. Evacuation times were determined

using the Interactive Dynamic Evacuation Model (IDYNEV), a macroscopic simulation model

that provides dynamic transportation routing features. The IDYNEV System that was employed

for this study is comprised of several integrated computer models. One of these is the PC-

DYNEV (DYnamic Network EVacuation) macroscopic simulation model that was developed by

KLD under contract with the Federal Emergency Management Agency (FEMA). KLD has

performed the ETE studies for 15 of the 18 Combined Operating License (COL) and Early Site

Permit (ESP) applications currently on file with the NRC. The North Anna project began in May

2007 and extended over a period of 7 months, then revised in 2008. The revision included a

refinement of the calculations performed in the original report.

The ETEs calculated for the North Anna Power Station are similar to the results obtained

utilizing RtePM. In the North Anna study, the trip generation time is calculated at 4 hours. Trip

generation in the NRC report is the value of the time that the last person entered the roadway.

RtePM evacuation times vary slightly when compared to the 2008 NAPS study due to the

“custom response time” that can be manipulated in RtePM. If the custom response time in

RtePM is set to “1”, meaning the population would start evacuating within one hour, the

evacuation time is shorter in RtePM than that of the NRC study. When the “custom response

time” in RtePM is changed to 4, meaning the last person entering the roadway is four hours later,

the evacuation times are comparable to that of the NAPS study. See Figure 5.

18

Figure 5 North Anna NRC Comparison

The North Anna comparison results in Figure 5 confirm the importance of accurately recreating

the scenario conducted by the NRC with the same data entered in to RtePM. System

terminology needs to be considered, such as custom response time for RtePM and trip generation

time for the NRC study as described above.

Trip generation times were obtained in the NRC study by consulting emergency managers in the

area, conducting surveys, and contacting the public using phone interviews. Also considered

was time of day, weather, number of vehicles, and schools in session. Emergency managers,

survey and telephone interview results provided different response times due to variables such as

school children riding busses home, spouses returning home from work and packing, shoveling

driveways, or picking up relatives. Although RtePM does not account for the time to shovel a

driveway, these variables can be defined in RtePM by adjusting response times, day or night

timeframe, number of people per vehicle, or changing the percentage of population clearing the

area. Evacuation end points were selected based on wind direction. If evacuees are located up-

wind of the power plant, they would evacuate to any end point away from the power plant. If

evacuees are located down-wind of the power plant, they would evacuate to end points which are

perpendicular to the wind direction.

2. Bellefonte Nuclear Plant ETE Report, September 2007

This report describes the analyses undertaken and the results obtained by a study to develop

ETEs for the proposed Bellefonte Nuclear Plant (BLN) located in Jackson County, Alabama.

0

5

10

15

20

25

30

0 30 60 90 120 150 180 210 240 270 300

NU

MB

ER O

F V

EHIC

LES

in T

HO

USA

ND

S

EVACUATION TIME IN MINUTES

Evacuation Time Estimate Summer, Midweek, Midday, Good Weather

2 mile RtePM 2 5 mile RtePM 5 10 mile RtePM 10

19

ETEs were also calculated using the IDYNEV simulation model. This project began in August,

2006 and extended over a period of 8 months. The traffic demand and trip-generation rate of

evacuating vehicles were estimated by consulting emergency managers in the area, conducting

surveys, and contacting the public using phone interviews. The trip generation rate reflected the

estimated mobilization time (i.e., the time required by evacuees to prepare for the evacuation

trip) that was computed using the results of the telephone survey of Emergency Planning Zone

(EPZ) residents. The evacuation time comparison results are displayed in Figure 6.

Figure 6 Bellefonte NRC Comparison

Results closely compare between the Bellefonte study and RtePM when comparing 2, 5, and

10 mile radius evacuation zones. Evacuees, number of vehicles, time of day, and custom

response time were compared. Evacuation end points were selected based on wind direction.

If evacuees are located up-wind of the power plant, they would evacuate to any end point

away from the power plant. If evacuees were located down-wind of the power plant, they

would evacuate to end points which are perpendicular to the wind direction.

3. Callaway Nuclear Plant ETE Report, September 2007

Population estimates for the Callaway evacuation zones in Ameren, Mo, were based upon

Census 2010 data. Estimates of employees who reside outside the EPZ and commute to

work within the EPZ were based upon data obtained from surveys of major employers in the

EPZ. Population estimates at special facilities were based on available data from county

emergency management offices and from phone calls to specific facilities. Roadway capacity

estimates were based on field surveys and the application of the Highway Capacity

Manual 2010. Population mobilization times are based on a statistical analysis of data

0

5

10

15

20

25

30

35

40

0 30 60 90 120 150 180 210 240 270

NU

MB

ER O

F V

EHIC

LES

in T

HO

USA

ND

S




20

acquired from a random sample telephone survey of EPZ residents. The relationship between

resident population and evacuating vehicles were developed from telephone surveys. Average

values of 2.40 persons per household and 1.35 evacuating vehicles per household were used.

All data was modified and duplicated in RtePM and number of vehicles and the time to

evacuate is displayed below:

Figure 7 Callaway NRC Comparison

Again the results are comparable between the Callaway study and RtePM when comparing 2, 5,

and 10 mile radius evacuation zones. Evacuees, number of vehicles, time of day, and custom

response time were compared. Evacuation end points were selected based on wind direction. If

evacuees were located up-wind of the power plant, they would evacuate to any end point away

from the power plant. If evacuees were located down-wind of the power plant, they would

evacuate to end points which are perpendicular to the wind direction.

4. Nine Mile Plant, ETE Report, September 2007

The Nine Mile Plant ETE report was selected for comparison in order to validate evacuation

times during adverse weather. The Nine Mile plant is located on the shores of Lake Ontario in

Lycoming, NY, and receives heavy lake effect snowfall during the winter months. The report

estimates roadway free-flow speed and capacity reductions of approximately 20 percent under

snow conditions. Transient population reductions are not assumed for snow scenarios since

tourism is not high at this time of year.

0

5

10

15

20

25

0 30 60 90 120 150 180 210 240 270

NU

MB

ER O

F V

EHIC

LES

in T

HO

USA

ND

S




21

Figure 8 Nine Mile Point NRC Comparison

Although RtePM does not directly model weather into the simulation, it does provide the

ability to simulate the effect that weather would have on evacuation clearance times by

changing scenario input data. Varying data inputs was accomplished easily in RtePM by

reducing roadway speed limit by 20 percent, reducing the number of lanes available, and/or

closing smaller roads which have not been cleared. By decreasing the free flow speed on

roads during snow or heavy rain events, RtePM clearance times will adjust accordingly. The

time to shovel driveways and clear parking lots can also be accounted for by adjusting the

response time in RtePM. Validation times are displayed in Figure 8.

SUMMARY: Actual evacuation time results of the studies conducted for the NRC compared

to RtePM evacuation times were very similar as evidenced by the near parallel lines for each

of the 2, 5 and 10 mile radius lines for the NRC study and the 2, 5 and 10 RtePM radius lines.

Variables considered were the number of commuters, employees of the plant, bussing of

school students, vacation population being evacuated (transients), and the number of

ambulatory evacuees. ETE’s were calculated using RtePM in less than thirty minutes, saving

enormous time and money compared to conventional methods that take up to 6 months.

3. (B) Event Validity - Comparison to Real World Assessments

The Event Validity validation technique was also used to compare RtePM to real world

assessments. In this technique, a rigorous comparison of M&S performance with real world

phenomena was conducted. The test and review process compared RtePM results with known or

0

5

10

15

20

25

0 30 60 90 120 150 180 210 240 270 288

NU

MB

ER O

F V

EHIC

LES

in T

HO

USA

ND

S


Evacuation Time Estimate Winter, Weekend, Midday, Snow


22

expected behaviors. This validation technique compares the M&S results to some authoritative

reference data that defines what the expected results should be.

DDL Omni Engineering compared random Post Storm Assessments (PSA) and Hurricane

Evacuation Studies (HES) with results compiled while operating RtePM. These studies looked at

the vulnerability of a population to hurricane threats and gave guidance to local emergency

managers planning an evacuation of the community.

PSAs provide actual clearance times based on interviews with emergency planners, local

officials, and telephone surveys of individuals who remained in place during the hurricane.

During interviews, different understandings of the meaning of "clearance time" existed, so the

actual clearance times reported are approximate, but the most reliable, up to date data available.

One of the key outputs of an HES process is the matrix of Evacuation Clearance Times - the

number of hours it takes to move the threatened population to safety given various factors such

as the category of storm, the tourist occupancy (or population) of the area at the time, and public

responsiveness. The HES provide estimated clearance times provided to FEMA by the Army

Corp of Engineers prior to a storm.

The following PSAs and HES were utilized in determining RtePM validity by comparing real

world assessments results to the model simulation.

Table 3 Real World Comparison

Data Compared PSA Actual Clearance

Time Experienced

RtePM Calculated Clearance

Time

HES Estimated Clearance

Time

Hurricane Hugo

Camden County, GA 7 HRS 8.2 HRS 6 HRS

Glynn County, GA 8.5 HRS 8.5 HRS 8 HRS

Hurricane Opal

Escambia County, FL 9 HRS 9.8 HRS 15.5 HRS

Santa Rosa County, FL 8 HRS 8.7 HRS 7.5 HRS

Mobile County, AL 10 HRS 9.8 HRS 14 HRS

Baldwin County, AL 8 HRS 9.1 HRS 14 HRS

Hurricane Floyd

Beaufort County, SC 24 HRS 18 HRS 20 HRS

Brunswick County, NC 5 HRS 8.5 HRS 8.5 HRS

New Hanover County, NC 8 HRS 10 HRS 7.5 HRS

Hurricane Andrew

Dade County, FL 13 HRS 17 HRS 15 HRS

23

Data Compared PSA Actual Clearance

Time Experienced

RtePM Calculated Clearance

Time

HES Estimated Clearance

Time

Broward County, FL 17 HRS On going 23 HRS

Hurricane Ike

Houston/Galveston, TX 36 HRS 25.5 HRS 19 HRS

Matagorda County, TX 10 HRS 8.4 HRS 8 HRS

Hurricane Bonnie

Georgetown County, SC 9 HRS 11.6 13 HRS

Horry County, SC 12.5 HRS 11.9 13 HRS

POST STORM ASSESSMENT SUMMARY

In all cases, RtePM can produce similar results to HES or PSA by adjusting input numbers

such as:

1. Population - Increasing or decreasing the amount of population evacuating varies the

evacuation time

2. People per vehicle - Increasing or decreasing the number of people per vehicle, varies

the evacuation time

3. Percent of population clearing area - Increasing or decreasing the percentage of people

clearing area, changes the evacuation time

4. Percent using shelters – The percentage of people using shelters affects estimated

evacuation time

5. Modify roads and endpoints – Modifying roads and destinations result in change of

evacuation times

6. Response rate – Changing custom response rates affect evacuation times

7. Seasonal population – Changes in tourist population

8. Level of Background traffic – changes in background traffic affect evacuation times

The purpose of this study is not to manipulate RtePM so that it matches the exact results of

other studies. The purpose of these real world comparisons was to gather as much real world

data from PSAs and enter that data into RtePM and compare the results. When the results do

not match exactly, which is the case in the table above; it is not our intent to make the results

match, even though by manipulating the above listed variables, RtePM can mimic the results

of other models.

Hurricane Hugo – In Georgia, clearance times calculated for FEMA/Corps studies compared

well with the actual times experienced in Hurricane Hugo. For those counties carrying out major

24

evacuations, RtePM study produced times were within an hour of actual times.5 RtePM

evacuation times were nearly identical to the actual and FEMA evacuation times.

Hurricane Opal - Mobile and Baldwin counties reported that traffic flowed fairly smoothly

through their in-county evacuation networks. They also reported that county residents eventually

met significant congestion and long delays north on Interstate 65 in Escambia County and

beyond. Low evacuation times compared to HES evacuation estimates indicate that a smaller

portion of the population evacuated than anticipated.6 RtePM evacuation estimates were nearly

identical for the Mobile and Baldwin County HES actual evacuation times when considering that

just fifty percent of the population cleared the area.

RtePM evacuation times were very acceptable when compared to actual evacuation times of

Escambia and Santa Rosa Counties. Sixty one percent of the population evacuated, and when

configuring RtePM with a sixty one percent of population clearing area, the results were nearly

identical. Escambia did not implement reverse lane strategies on its evacuation routes because of

the resources that would be necessary to control access at intersections were not available.7

Reversible lanes were not used in the RtePM calculations in order to closely simulate actual

events.

The evacuation out of Santa Rosa County was described by county officials as a "nightmare."

The general evacuation was underway at approximately 8 am. Roads feeding Interstate 10

backed up as construction on the Interstate slowed traffic. RtePM calculations included a

reduction of “free flow Speed’ when calculating evacuation times along Interstate 10. Several

in-county back roads were flooded due to heavy rainfall that occurred during the week preceding

Opal. We simulated this phenomenon by closing all small arterial end-points and evacuating

along major arterial and highways.

Hurricane Floyd - The path of Hurricane Floyd and the uncertainty which existed regarding its

potential landfall, caused massive evacuations of populations in the Florida, Georgia, South

Carolina, and North Carolina coasts. These massive evacuations that occurred as a result of the

uncertainty regarding anticipated landfall of Hurricane Floyd, led to the use of various traffic

control actions by local and state officials. Most counties experienced flooding as the main

problem encountered during the evacuation. Due to flooding, the amount of evacuating

population may have been reduced resulting in the Brunswick County five hour observed

evacuation time.8 RtePM, actual and FEMA evacuation times were comparable.

5 Hurricane Hugo Assessment, January 1990

6 Hurricane Opal Assessment, October 1995

7 Hurricane Opal Assessment, October 1995

8 Hurricane Floyd Assessment, May 2000

25

Hurricane Andrew – The Dade and Broward County actual evacuation times were slightly less

than calculated times. This was directly due to less participation on the part of the residents than

had been assumed in the modeling. The times also suggest that many evacuees may have left the

area well in advance of the official evacuation order given early Sunday morning. Traffic counts

collected from the permanent count stations that were functioning prior to and during the

Andrew evacuation confirm this phenomenon.9 While attempting to run the Broward county

scenario in RtePM an, “Error configuring scenario to run” message is displayed. Initial research

conducted by DDL Omni Engineering is inconclusive as to the cause of the error. Further

research is being conducted. When attempting to run Dade County scenario, we were not able to

select the entire county for evacuation. By design, RtePM allows for the selection of 1500

census ID blocks. Dade County exceeds this limit, resulting in “the total number of population

blocks selected has exceeded the maximum of 1500, please retry selection” error message. This

was easy to overcome by running the evacuation in two phases. By splitting the county in half,

the scenario ran perfectly and the evacuation times were comparable.

Hurricane Ike - The 2010 USACE HES updated clearance times were generated based on

differing intensity strengths of hurricanes, levels of background traffic, and the rapidity of

response by evacuees, and different tourist occupancy levels. Category 3 levels were used for

this comparison.10

Hurricane Bonnie - Interviews and analysis conducted for the post Bonnie effort revealed modest

evacuation participation rates on the part of the permanent population. Shelter usage was low

except in Horry County, South Carolina, where many tourists went to public shelters. Few traffic

problems were reported. The lack of traffic problems indicates that local and state officials

started the evacuation in a timely manner that traffic control was appropriate and effective, and

that participation rates were much less than the 100% rates used in the study calculations.

RtePM evacuation times were closer to actual evacuation times than FEMA predicted evacuation

times.

4. Internal Validity Stability - Reliability

A good model/simulation must give the user confidence that the results are accurate. This

confidence must be gained in some part by model reliability. Model reliability is the fourth

validation technique that was conducted. If the user simulates an evacuation several times

without any changes in the scenario, it would be reasonable for the user to expect the

evacuation times would remain the same. When running the same scenario multiple times, the

user should expect the exact same results. The stability/reliability testing of RtePM was

satisfactory and Table 4 represents a sample of the testing results.

9 Hurricane Andrew, January 1993

10 Post Storm Assessment: Hurricane Ike, June 2010

26

Table 4 Reliability Confidence

Location Evacuation Time #1 Evacuation Time #2 Evacuation Time #3

Bethany Beach, DE 14.3 Hours 14.3 Hours 14.3 Hours

Bethesda, MD 8.1 Hours 8.1 Hours 8.1 Hours

Mobile, AL 16.4 Hours 16.4 Hours 16.4 Hours

Panama City, FL 10.7 Hours 10.7 Hours 10.7 Hours

Galveston, TX 25.6 Hours 25.6 Hours 25.6 Hours

Hancock County, MI 8.3 Hours 8.3 Hours 8.3 Hours

Washington D.C. 16.2 Hours 16.2 Hours 16.2 Hours

Anaheim, CA 4.7 Hours 4.7 Hours 4.7 Hours

Concord, CA 8.1 Hours 8.1 Hours 8.1 Hours

Virginia Beach, VA 23.7 Hours 23.7 Hours 23.7 Hours

Savannah, GA 8.4 Hours 8.4 Hours 8.4 Hours

SUMMARY: When duplicating scenarios and running the scenario multiple times, the

evacuation times in Table 4 remained the same, indicating model/simulation reliability. The

following data sets were tested:

1. Population

2. People per vehicle

3. Percent of population clearing area

4. Percent using shelters

5. Modify roads and endpoints

6. Response rate

7. Seasonal population

8. Level of Background traffic

5. Parameter Variability The fifth validation technique used was the Parameter Variability-Sensitivity Analysis

technique. It consisted of changing RtePM input and initial condition parameters to determine

the effect upon the model and its output. By running RtePM scenarios several times and

changing minor inputs to the scenario, we were able to determine that results produced by

27

RtePM are consistent with parameter changes that were made. Table 5 exhibits the results

when varying parameter inputs are manipulated.

Table 5 Parameter Variability

Parameter Input Change To Parameter Measurement Method Results

Population Change

Percent

Alter the percent of

population clearing the

area in:

1. Baldwin County, AL

2. Las Vegas, NV

3. Virginia Beach, VA

4. Portland, OR

Test for accurate

change in evacuation

time when changing

population density by

adjusting the

population change

block (%)

All tests were

satisfactory. When

the % of

population was

altered, the time

to evacuate

changed

proportionately.

Seasonal Scalability Alter the seasonal

population clearing the

area in:


2. Las Vegas, NV


4. Portland, OR

Test for accurate


time when changing

seasonal population.

All tests were

satisfactory.

When the seasonal

population was

altered, the time

to evacuate

changed

proportionately.

Number of

evacuees per

vehicle

Alter the number of

evacuees per vehicle

when clearing the area in:


2. Las Vegas, NV


4. Portland, OR

Test for accurate


time when adjusting

people per vehicle

All tests were

satisfactory. After

altering the # of

people per vehicle,

evacuation times

changed

proportionately.

Use of reversible

lanes / Contraflow

Alter reversible lanes

clearing the area in:


2. Las Vegas, NV


4. Portland, OR

Build scenarios with

reversible lanes and

calculate evacuation

times for different

scenarios

All tests were

satisfactory.

When contra-flow

was utilized,

evacuation times

decreased.

28

Parameter Input Change To Parameter Measurement Method Results

Road closures /

includes bridges,

tunnels

Alter the road closures /

including bridges, tunnels

clearing the area in:


2. Las Vegas, NV


4. Portland, OR

Close roads, bridges or

tunnels and determine

if evacuation times

differ

All tests were

satisfactory. When

roads were closed,

evacuation times

increased.

Background Traffic/ Background traffic simulates vehicles using the roadways but not directly participating in the evacuation.

Alter the background

traffic density clearing

the area in:


2. Las Vegas, NV


4. Portland, OR

Change background

traffic density and

determine if

evacuation times differ

All tests were

satisfactory.

When background

density was

altered,

evacuation times

changed

proportionately.

29

The following tables show results of random population and seasonal population changes in

different cities and the corresponding results. We expected that as population increased by an

operator selected percentage or seasonal increase in population, evacuation times would

increase.

Table 6 Change in Population and Seasonal Scalability Comparisons

Baldwin County, AL Background Traffic RtePM 25% Population Increase Seasonal Population

Evacuation Time: Hours

None 15.2 17.9 16.3

Low 15.5 18.8 16.4

Medium 15.8 18.8 16.4

High 15.8 19.8 17.2

Portland, OR Background Traffic RtePM 25% Population Increase Seasonal Population


None 9.5 11.5 11.6

Low 10.0 11.8 12.4

Medium 10.4 12.0 12.5

High 10.5 12.4 12.6

Las Vegas, Nevada Background Traffic RtePM 25% Population Increase Seasonal Population


None 11.5 13.8 15.2

Low 11.9 14.2 15.4

Medium 12.1 14.4 15.4

High 12.2 14.5 15.7

Virginia Beach, VA Background Traffic RtePM 25% Population Increase Seasonal Population


None 13.7 15.8 23.8

Low 13.8 16.0 23.8

Medium 13.9 16.6 23.8

High 14.1 16.8 23.8

Finding: When selected cities were tested for change in population, evacuation times changed

proportionately.

30

The following tables show results for change in number of people per vehicle. The test was

conducted using all selectable types of background traffic. We expected that as the number of

people per vehicle increased, evacuation times would decrease and vice versa.

Table 7 Traffic Density (PPV and Background Traffic) Comparisons

Baldwin County, AL Background Traffic 2.5 PPV 1 PPV 4 PPV


None 16.3 34.5 12.2

Low 16.4 34.8 12.3

Medium 16.4 34.8 13.0

High 17.2 35.8 13.0

Portland, OR Background Traffic 2.5 PPV 1 PPV 4 PPV


None 9.5 20.4 8.4

Low 10.0 20.7 8.4

Medium 10.4 21.3 8.4

High 10.5 21.4 8.5

Las Vegas, NV Background Traffic 2.5 PPV 1.5 PPV 4 PPV


None 11.5 17.9 8.5

Low 11.9 18.2 8.6

Medium 12.1 18.3 8.8

High 12.2 18.3 8.9

Virginia Beach, VA Background Traffic 2.5 PPV 1.5 PPV 4 PPV


None 23.8 30.8 19.8

Low 23.8 30.8 19.8

Medium 23.8 30.8 19.8

High 23.8 30.8 19.8

Finding: When selected cities were tested for change in people per vehicle and background

traffic, evacuation times changed proportionately.

31

The following tables show results for changes in Road Closure and Contra Flow. The test was

conducted using four levels of background traffic. We expected that evacuation times would

increase as roads were closed and evacuation times would decrease if contra flow techniques

were used.

Table 8 Road Closures and Contra Flow Comparisons

Baldwin County, AL Background Traffic Basic Road closure AL50 North near Loxley Contra Flow


None 16.3 18.9 15.3

Low 16.4 18.9 15.9

Medium 16.4 18.9 16.0

High 17.2 19.4 16.3

Portland, OR Background Traffic Basic Road closure I-5 North bound Contra Flow


None 9.5 12.5 9.4

Low 10.0 13.5 9.4

Medium 10.4 13.5 9.4

High 10.5 13.6 9.4

Las Vegas, NV Background Traffic Basic Road closure I-515 South bound Contra Flow


None 11.5 14.3 11.0

Low 11.9 15.0 11.2

Medium 12.1 15.2 11.2

High 12.2 15.3 11.4

Virginia Beach, VA Background Traffic Basic Road closure I-664 North bound Contra Flow


None 27.7 29.7 25.6

Low 27.9 29.7 25.6

Medium 28.0 29.7 25.9

High 28.6 29.7 26.0

Finding: When selected cities were tested for road closures and contra flow, evacuation times

changed proportionately.

32

SUMMARY: Parameter Variability was tested by running 144 scenarios that implemented

minor changes to the scenario inputs. Different levels of background traffic were used in all

scenarios, simulating vehicles using the roadways but not directly participating in the

evacuation. Testing included changes to population, seasonal population, background traffic,

people per vehicle, road closures and reversible lanes. Parameter Variability testing confirms

that RtePM is reliable and stable when changing input parameters and confirming expected

evacuation time results, as indicated in the Tables 6,7, and 8. Results are listed below:

• Population – When increasing population by 25%, the time to evacuate increased

proportionally due to the increased number of vehicles on the road. In the Baldwin

County, AL example, the evacuation time increased from 15.2 to 17.9 with a 25%

increase in population.

• Seasonal population – When adding seasonal population, the number of vehicles on the

roadways increased, therefore increasing evacuation.

• Background traffic – When background traffic increased, evacuation time increased

due to the increase in vehicles on the roadway.

• People per vehicle – As number of people per vehicle increased, evacuation time

decreased due to the fewer amount of vehicles on the road. As number of people per

vehicle decreased, evacuation time increased because there are more vehicles on the

road.

• Road closures and reversible lanes – Evacuation times increased as roads were closed

in all simulations. When adding contra-flow lanes, the evacuation times decreased in

all cases.

6. Historical Data Validation

JHU/APL validated RtePM utilizing the Mississippi Hurricane Evacuation Study, of April

2012 conducted by the US Army Corp of Engineers (USACE). This comparison study was

used as baseline against which the RtePM output was compared. To accomplish this, a

representative sampling of counties from the USACE study was selected and the RtePM tool

was used to create scenarios that were equivalent for comparisons to the USACE model

results. RtePM model parameters were adjusted to closely match the USACE model

parameters such as the evacuation response curves.

The first scenario tested was for Hancock County with maximum occupancy and people per

vehicle was set to 2 to match the USACE study. The end points for this comparison were in

all directions. The results are shown in Figure 9.

33

Figure 9 Hancock County Comparison

For the slow and medium evacuation response curves, RtePM results from JHU/APL and DDL

Omni Engineering closely match the USACE study. For the fast and immediate evacuation

response, the RtePM model run by DDL Omni Engineering yielded evacuation times that were

closer to the USACE study. The second comparison was for Jackson County with maximum

occupancy, 2 people per vehicle, and end points in all directions. The results are shown in

Figure 10.

11

8

6

6

11.1

8.1

5.2

1.3

11.3

8.3

5.3

3.1

0 2 4 6 8 10 12

Slow

Medium

Fast

Immediate

Hours to Evacuate

R

e

s

p

o

n

s

e

Hancock County

DDL OMNI

JHU-APL

USACE

34

Figure 10 Jackson County Comparison Scenario A

For all evacuation response curves the RtePM model yielded evacuation times that are shorter

than the USACE study and slightly higher than the JHU/APL study. RtePM performed well and

evacuation times are reliable.

For the next comparison, Jackson County was used again; however the end points for the

evacuation were restricted to travel east and north out of the evacuation zones. All other

parameters were the same. The results are shown in Figure 11.

12

10

7

7

11.2

8.2

5.3

4.3

11.4

8.4

6.2

5.2

0 2 4 6 8 10 12 14

Slow

Medium

Fast

Immediate

Hours to Evacuate

R

e

s

p

o

n

s

e

Jackson County Scenario A

DDL OMNI

JHU-APL

USACE

35

Figure 11 Jackson County Comparison Scenario B

In this instance the results yielded by DDL Omni Engineering were starkly shorter for all

evacuation response curves. The smallest difference is for the fast evacuation response where

the difference was 6 hours and 12 minutes. The largest difference is for the medium evacuation

response where the difference was 9 hours and 18 minutes.

SUMMARY: Based on these comparisons, it is clear that when compared to the USACE study,

RtePM results yield shorter forecast evacuation times in nearly all cases. However, the

forecasted difference for slow and medium evacuation response curves is not significantly

different when both models use evacuation routes out of the evacuation zone(s) in all directions.

This comparison shows that for fast and immediate evacuation response curves or when the

model scenario is designed to restrict traffic to specific directions, the RtePM model will likely

yield evacuation times that are significantly shorter for all response curves.

22

20

17

17

12.2

10.7

10.8

9.8

11.4

8.4

7.3

6.3

0 5 10 15 20 25

Slow

Medium

Fast

Immediate

Hours to Evacuate

R

e

s

p

o

n

s

e

Jackson County Scenario B

DDL OMNI

JHU-APL

USACE

36

JHU/APL also conducted three event validation studies in the following areas:

• Houston/Galveston

• Delaware

• Ocean City, Maryland

DDL Omni Engineering attempted to validate the JHU/APL studies to determine the quality of

their event validation. While attempting to duplicate the results of the JHU/APL studies, it was

discovered that there was not enough input data set information available for DDL Omni

Engineering to perform event validation in the Houston/Galveston or Delaware area. Numerous

attempts were made to duplicate and run the scenarios, but the results were unreliable. Additional

research and discussion with JHU/APL is needed to ensure the same JHU/APL parameters are

used when validating their results.

DDL Omni Engineering was able to perform testing using data available for Hurricane Irene in

the Ocean City, Maryland coastal region. The University of Maryland (UMD) Traffic Safety &

Operations Laboratory, A. James Clark School of Engineering, in partnership with the Maryland

Department of Transportation, State Highway Administration, pre-deployed vehicle counting

devices at specific locations along hurricane evacuation routes leading away from Ocean City,

Maryland. The results of these data counters are displayed in the UMD Count of Tables 9, 10,

and 11.

The results of JHU/APL RtePM model are displayed in the JHU/APL RtePM column. DDL

Omni Engineering validation results are displayed in the DDL RtePM column.

Three pre-deployed traffic counters that would produce traffic counts by hour at logical

evacuation end points were used:

• Ocean City Expressway (Rt. 90)

• Coastal Highway (Rt. 1)

• Ocean Gateway (Rt. 50)

The figures below show the cumulative traffic counts at these three specific locations by each

hour from the start of the evacuation for both the actual traffic data and RtePM. The x-axis is

hours and the y-axis is cumulative traffic count.

A. Ocean City Expressway - The evacuation times are nearly identical at the eighteen hour mark

as shown in Table 9 and Figure 12. The difference between the RtePM evacuee numbers and

the actual numbers can be attributed to human behavior/what time the evacuees decided to

evacuate.

37

Table 9 Ocean City Express Way (RT90) Vehicle Count

Hour JHU/APL RtePM DDL RtePM UMD Count

1:00 67 60 298

2:00 119 136 525

3:00 264 332 785

4:00 416 672 821

5:00 856 1286 881

6:00 1605 1984 1022

7:00 2343 2677 1368

8:00 3106 3393 1722

9:00 3843 4092 2105

10:00 4601 4802 2531

11:00 5338 5489 3021

12:00 6099 6192 3502

13:00 6550 6848 3964

14:00 6713 7238 4511

15:00 6855 7462 5164

16:00 6908 7548 5887

17:00 6993 7618 6508

18:00 6994 7624 6942

Figure 12 Ocean City Express Way (RT90) Graph

38

B. Coastal Highway (RT1) - The weighting of endpoints plays an important factor in the

analysis of the Coastal Highway shown in Table 10 and Figure 13. There is not enough

information about the weighting of endpoints in the JHU/APL reports for DDL Omni

Engineering to determine validity. If DDL Omni Engineering increases endpoint

weighting for the Coastal Highway, we can duplicate the road counter and JHU/APL

results, but there is not enough data to make a fair evaluation. More research is needed.

Table 10 Coastal Highway (RT1) Vehicle Count


1:00 83 76 201

2:00 209 145 430

3:00 363 244 791

4:00 680 328 1209

5:00 1021 412 1652

6:00 1344 496 2063

7:00 1693 580 2513

8:00 2014 664 2982

9:00 2365 748 3417

10:00 2688 832 3726

11:00 3037 916 3960

12:00 3358 1000 4098

13:00 3709 1084 4192

14:00 4028 1168 4193

15:00 4209 1252 4196

16:00 4317 1331 4210

17:00 4394 1412 4234

18:00 4395 1413 4259

39

Figure 13 Coastal Highway (RT1) Graph

C. Ocean Gateway (Rt. 50) - The difference between the RtePM evacuee numbers and the

actual numbers can be attributed to human behavior/what time the evacuees decided to

evacuate. Model results are only as accurate as the input that goes in the model, and

human behavior is hard to predict. The number of people that decided to evacuate, and

when they evacuated represent the difference in the results of Table 11 and Figure 14.

Table 11 Ocean Gateway (RT50) Vehicle Count


1:00 68 75 871

2:00 119 149 1754

3:00 276 315 2832

4:00 495 571 4289

5:00 852 995 5632

6:00 1568 1913 6882

7:00 2758 3387 7676

8:00 4612 5413 8401

9:00 6749 7709 8776

10:00 8692 9920 9060

11:00 9980 11779 9287

12:00 10788 13188 9509

13:00 11204 13847 9690

14:00 11431 14136 9798

15:00 11581 14324 9866

16:00 11658 14423 9926

17:00 11732 14512 10816

18:00 11735 14525 11604

40

Figure 14 Ocean Gateway (RT50) Graph

Parameter-variability testing using background traffic and endpoint weighting was also

conducted in Baldwin county Alabama by JHU/APL.

Table 12 displays a comparison between JHU/APL data and DDL Omni Engineering data

when changing background traffic in RtePM. The results are satisfactory. There is a slight

difference in the results due to the fact that the exact parameters used in the JHU/APL scenario

were not available in our testing, inputs may have varied slightly.

Table 12 Background Traffic

EVACUATION TIMES FOR BALDWIN COUNTY AL

Background Traffic JHU-APL Evacuation Time DDL Omni Evacuation Time

None 16.1 hours 16.3 hours

Low 16.6 hours 16.4 hours

Medium 16.7 hours 16.4 hours

High 17.5 hours 17.2 hours

Table 13 displays a comparison between JHU/APL data and DDL Omni Engineering data

when changing End Point weighting. The results are satisfactory. There is a slight difference

in the results due to the fact that the exact parameters used in the JHU/APL scenario were not

available in our testing, inputs may have varied slightly.

41

Table 13 End Point Weighting

VALIDATION OF EVACUATION TIME WHEN CHANGING END POINT WEIGTING BALDWIN COUNTY AL

Basic Scenario Evacuation time

I-65 End Point Weighted at 70%



Evacuation time: Hours

Evacuation time: Hours

Evacuation time Hours

JHU DDL JHU DDL JHU DDL JHU DDL

16.1 16.3 18.6 18.3 19.6 19.3 20.6 20.3

End Point % of Evacuating

Vehicles

% of Evacuating Vehicles



JHU DDL JHU DDL JHU DDL

US-31 12.1 7.5 10.8 7.0 9.9 5.6 7.7

Muscogee Road

2.3 1.6 1.4 1.3 1.1 1.1 1.0

AL-59 8.5 4.2 2.0 3.8 1.6 2.4 0.9

I-65 67.0 79.4 79.0 81.9 81.9 85.6 85.8

CR-47 0.9 0.4 0.4 0.4 0.3 0.3 0.3

I-10 9.3 6.9 6.4 5.6 5.2 5.0 4.4

JHU/APL conducted a Face Validation study by consulting with SMEs to determine if the model

seemed reasonable to people who were knowledgeable about evacuation planning. The

following comments were documented in the RtePM Comparison Study conducted by Johns

Hopkins University, Applied Physics Laboratory, May 2012.11

• Brandon Bolinski, Alabama Emergency Management: The tool appeared at first to have

many steps, but after walking through the steps, it was found to be user friendly and

ultimately very intuitive. Roy Dunn concurred with Brandon’s comment about the user

friendly aspect of the tool as well as the decision to operate the tool on the ESRI Flex

platform. John Eringman commented as to the speed of the tool; with David George of

JHU/APL noting that the speed is a factor and we are working to enhance that

functionality. John was particularly interested in conducting a full county and multiple

county evacuations for Alabama with it.

• FEMA Region III, Regional Advisory Council: Value of RtePM is that it could predict

numbers of evacuees into the area.

11 Real Time Evacuation Planning Model (RtePM) Comparison Document, JHU/APL, May 2012

42

• Department of Homeland Security (DHS) Science and Technology (S&T) 1st Responder

Working Group: Overall you have done a miraculous job developing this model. It is

easy to use and gives hr by hr results.

• The National Emergency Management Association (NEMA) will support the

development, maintenance and training for the RtePM and formally endorse the product.

RtePM Focus Group, National Hurricane Program (NHP) – ICCOH with NEMA

Subcommittee

• Cecil County, Maryland Department of Emergency Services: Excellent, real world results

and considerations.

• Pacific Northwest National Laboratory: This can be a very valuable tool - especially for

smaller localities that will not have financial resources to develop such a concept on their

own. Also think the concept of a "shared/standardized" knowledge base will be valuable

- much like HURREVAC makes it easy for folks not based locally to see/use/collaborate

if information and platform standard.

• NEMA Response and Recovery Committee: Overall, an exciting and interesting tool.

However, to be applicable to a highly urbanized environment such as New York City

there must be a way to incorporate public transit information, and to account for car

ownership of less than one car per household.

• Homeland Security Infrastructure Foundation-Level Data (HIFLD) Staff: Additionally, I

did create scenarios for the Delmarva, and for our two nuclear power plants for

comparison purposes to existing studies, and the numbers verified quite well against the

numbers on file in our existing studies! That alone provided a higher confidence in the

results. I’ll look into some more items for suggestions, but again this is a very good

move forward.

• Virginia Department of Emergency Management: This is an easy to use tool that is very

intuitive to use. I like the interface and the mapping. If this tool is made available for

future use, there needs to be commitment that the information would be a current as

possible.

• DHS S&T Evacuation Symposium: Really good tool for special evacuations – Nuclear

power plants, dams, wild land fires, venue applications, etc.

• Office of Emergency Management, San Diego County, California: Strength of tool -

modeling displays choke points, indicated how / where / when.

43

DOCUMENT SUMMARY

The purpose of this report is to provide results of the Independent Verification and Validation

(IV&V) of the Real Time Evacuation Planning Model (RtePM). Focus was centered on ensuring

RtePM meets system requirements and fulfills its intended purpose by employing Conceptual

and Operational Validation Techniques.

CONCEPTUAL VALIDATION

DDL Omni Engineering SME’s took the approach to identify what an evacuation

model/simulation should have in order to be an effective evacuation tool. We called this

conceptual validation and the following steps were accomplished in order to determine whether

RtePM had the features of an effective evacuation tool:

1. Identified the purpose of RtePM and the intended users of the model/simulation.

2. Defined critical features that an evacuation planning model must have in order to meet

the purpose of the planning model. DDL Omni Engineering identified the eight critical

features below:

a. Type of disaster- Determine evacuation time for any type of disaster.

b. Population density- The model must accurately model populations.

c. Roadway configuration- The model must accurately display roads and routes.

d. Traffic Density- The model must consider the effects of congestion.

e. Time to Evacuate- The model must accurately determine evacuation time.

f. Emergency Services- Should model the effects of emergency services.

g. Capabilities- The model/system capabilities and ease of use.

h. Weather- The model should consider the impact of weather.

3. Developed essential elements or requirements of a planning model to make the critical

features function properly.

Our Conceptual validation findings determined that RtePM meets all the critical features that an

effective evacuation planning model/simulation should have except considering the impact of

weather and the effects of emergency services. However, we believe workarounds such as

changing speed limits, or closing roads/bridges/tunnels can simulate the adverse effects of

weather. Locations of emergency services such as Police and Fire stations would be beneficial.

The results of RtePM’s ability to meet the critical features and essential elements are

documented in Appendix B.

44

OPERATIONAL VALIDATION

In accomplishing Operational validation, we used the following techniques to develop model

confidence. Each of these techniques has specific application criteria that were used as a

guideline for model validation.

1. Data Validity- This validation technique was used to validate the population and roadway

facilities databases in RtePM. Using this technique, DDL Omni Engineering found that

the critical features “population” and “ roadway configuration” are modeled accurately in

RtePM data.

a. Population- RtePM population data was identical to U.S. Census/LandScan

population data for the seven U.S. cities and the thirty-five census ID blocks

sampled.

b. Roadway Configuration- We selected fifteen well known roads and routes in the

Hampton Roads area to evaluate and found all fifteen to be accurately portrayed

in RtePM. In twenty-nine of the thirty instances tested outside of Hampton

Roads, RtePM maps and data accurately compared to Google Maps. In one of the

thirty instances tested, the road model data was inconsistent with the RtePM map

and Google Maps and may affect evacuation time results. DDL Omni

Engineering recommends the RtePM ‘Roads Tab’ be updated by local emergency

managers who are familiar with local roads and routes

2. Comparison to other models- Comparison of other evacuation models with RtePM was

not successful due to time constraints, accessibility, complexity, and the lack of

comparable capabilities. The resulting validation would be suspect, with a reduced level

of confidence in the accuracy of the comparisons. Comparison of the RtePM model to

other models was not accomplished

3. Event Validity- This validation technique evaluated the results of running the RtePM

model/simulation to real world results, e.g. model/simulation evacuation times were

compared to actual PSA evacuation times for hurricanes or the NRC evacuation case

studies.

a. NRC- Evacuation case studies for the North Anna, Bellefonte, Callaway, and

Nine-Mile nuclear power plants were used to compare evacuation times with

RtePM. NRC evacuation time results were comparable to the evacuation times of

RtePM.

b. PSA- The following PSA were compared to RtePM:

i. Hurricane Hugo- Clearance times calculated for FEMA/Corps studies

compared well with the actual times experienced in Hurricane Hugo.

ii. Hurricane Opal-.RtePM evacuation times were acceptable when

compared to actual evacuation times of Escambia and Santa Rosa

Counties.

45

iii. Hurricane Floyd- The path of Hurricane Floyd and the uncertainty

which existed regarding its potential landfall, caused massive

evacuations of populations in the Florida, Georgia, South Carolina, and

North Carolina coasts. RtePM, actual and FEMA evacuation times

were comparable.

iv. Hurricane Andrew- The Dade and Broward County actual evacuation

times were slightly less than RtePM calculated times.

v. Hurricane Ike- The 2010 USACE HES updated clearance times were

generated based on differing intensity strengths of hurricanes, levels of

background traffic, and the rapidity of response by evacuees, and

different tourist occupancy levels. RtePM calculated times compared

favorably to the 2010 USACE HES times.

vi. Hurricane Bonnie- RtePM evacuation times were closer to actual

evacuation times than FEMA predicted evacuation times.

4. Internal Validity- Several runs of the model were made to determine the amount of

(internal) variability in the model. When duplicating scenarios and running the scenario

multiple times, the evacuation times remained the same, indicating model reliability.

5. Parameter Variability-Sensitivity Analysis- This validation technique changed the

model’s input and initial condition parameters to determine the effect upon the model and

its output. Parameter Variability was tested by running 144 scenarios that implemented

minor changes to the scenario inputs. Testing included changes to population, seasonal

population, background traffic, people per vehicle, road closures and reversible lanes.

Parameter Variability testing confirms that RtePM is reliable and stable when changing

input parameters.

6. Historical Validation- Using this validation technique we validated the efforts that have

already been performed by the original developer. The data from the JHU/APL

Comparison Document, dated May 2012, was validated by DDL Omni Engineering for

this report.

The current method of developing evacuation studies may take months to complete, resulting in

expensive, possibly inaccurate plans that are outdated upon completion. With RtePM, a planner

can quickly modify a scenario, and see modified results within minutes. Planners can view,

understand, and manage evacuation plans as changes in the evacuation plan become necessary,

or to quickly create and examine alternatives.

As a part of the validation process, we ran large, medium, and small scale scenarios. When

simulating a small geographic area, new calculations are completed within seconds. A

simulation for a densely populated area can take up to two hours. For example, when

46

calculating for an area with a population of nearly two million, and seventy five percent of them

evacuating, the total run time to predict the evacuation time was about two hours.

The affordability of RtePM should also be considered. RtePM uses open source software, it's

Web-based, and it uses population and road-network data sets that local agencies can access for

free. Evacuation planners can update their emergency plans more frequently for less cost than

conventional evacuation studies that may take months to complete.

47

RECOMMENDATIONS

DDL Omni Engineering recommends further development of RtePM. When compared to other

evacuation models in use today, it has proven to be a fast, reliable, and easy to use system. The

observations listed below were compiled by DDL Omni Engineering during the IV&V process,

and should be considered for future development. These recommendations in no way diminish

the effectiveness or validity of the current version of RtePM:

• Recommend developing a planning window that is clear of the map when building a

scenario. An evacuation planning wizard interface would be useful down the left hand

side to show user progress of planning while map is still visible.

• Reversible flow function is difficult to operate. The reversible lane function needs to

be modified for ease of operation. When selecting a large section of highway for

reversible lanes, multiple road segments need to be selected. For example, Hampton

Roads to Richmond required approximately 50 road segments to be selected.

• Need to ensure that roads are easily updated on a yearly basis due to new construction.

• Update User’s Guide. Help window with user’s guide embedded in tool may be useful.

• Include Mass Transit in future builds. Not everyone owns a car in the National Capital

Region. They rely on mass transit.

• Configurable shelter location should be automated. Shelters may be added to the

evacuation area using the “Add” icon, but known shelters could be included in the

software, including capacity sizes. The number of people sheltering in place will affect

roadway congestion and evacuation times

• Although RtePM is not designed to consider weather, weather factors are important in

order to model dispersion direction, specifically wind direction and speed. This would

be critical in the case of wild fire evacuation planning, where smoke from the fire could

reduce visibility for driving in one direction, and not the other. Another instance where

wind direction would be critical is in the case of a dirty-bomb. Wind direction and

speed would dictate the area to be evacuated, direction of escape, or the need to shelter

in place. Currently the effects of weather can be simulated with workarounds such as

speed limits, road/bridge closures and people per car.

• Flood zone overlay would be useful in planning for evacuation. Enhanced topography

needs to be included. Terrain features that adequately identify potential flood zones

leading to road closures would be an enhancement that could be considered. In the

case of tidal surges during hurricanes, many roads are known to be underwater.

48

REFERENCES

A. Real Time Evacuation Planning Model (RtePM) Independent Validation and Verification

Plan, 16 October 2012

B. Sargent, R.G. (2010) “Verification and Validation of Simulation Models,” in the

Proceedings of the 2010 Winter Simulation Conference

C. National Research Council. 2012. Assessing the Reliability of Complex Models:

Mathematical and Statistical Foundations of Verification, Validation, and Uncertainty

Quantification. Washington, D.C.: The National Academies Press.

D. Department of Defense Standard Practice. Documentations of Verification, Validation,

and Accreditation (VV&A) for models and simulations MIL-STD-3022. 28 January

2008.

E. Real Time Evacuation Planning Model (RtePM) Comparison Document, May 2012;

Prepared by John Hopkins University, Applied Physics Laboratory.

F. Development of Evacuation Time Estimate Studies for Nuclear Power Plants, January

2005; Prepared by L.J Datson and J. Jones, Sandia National Laboratories, P.O. Box 5800,

Albuquerque, NM 87185; http://www.nrc.gov/reading-rm/doc-

collections/nuregs/contract/cr6863/cr6863.pdf

G. Maryland Hurricane Evacuation Study, December 1990; Prepared for Maryland

Emergency Management Agency FEMA Region III and U.S. Army Corps of Engineers

(USACE), Baltimore District; Prepared by Baltimore District, USACE under direction of

Colonel Frank R. Finch.

H. Mississippi Hurricane Evacuation Study Transportation Analysis, Final Report, April

2012; Prepared by Mobile District, USACE.

I. Army Corps of Engineers Technical Guidelines for Hurricane Evacuation Studies

September 1995; Prepared by USACE Wilmington District’s Flood Plan Management

Services Branch.

J. Technical Report, Dynamically Modeling Hurricane Evacuation Decisions, June 2007;

International Hurricane Research Center, Florida International University;

http://www.ihrc.fiu.edu, 11200 SW 8th Street, University Park, MARC 360

Miami, Florida 33199

http://www.ihrc.fiu.edu/

49

K. Modeling and Simulation of Staged Evacuations: A Case Study of Hurricane

Evacuations of Galveston Island; Xuwei Chen, Department of Geography, Northern

Illinois University, Dekalb, Ill. 60115

L. Hurricane Evacuation in Delaware, University of Delaware; REU Program Summer

2008 Disaster Research Center Science and Engineering Scholars, University

Transportation Center; Sarah Dalton, University of Delaware. http://www.ce.udel.edu/UTC/Dalton_Summer08

M. 2010 US Census Report; U.S. Department of Commerce, U.S. Census Bureau;

http://www.census.gov/2010census/

N. Virginia Hurricane Evacuation Guide, 2011; Commonwealth of Virginia, Department

of Transportation; 10501 Trade Court, Richmond, Virginia 23236,

http://www.vaemergency.gov/news/news-releases/2012/hurricane-guide

O. Dominion Power, North Anna 3 Combined License Application, Emergency Plan,

November 2007.

P. Bellefonte Nuclear Plant Development of Evacuation Time Estimates, September

2007; http://pbadupws.nrc.gov/docs/ML0731/ML073110546. ; KLD Associates Inc. 48

Mall Drive, Suite 8, Commack, NY 11725; Mr. Jay Maisler, Enercon

Corporation,14502 North Dale Mabry Hwy, Suite 200, Tampa, FL 33618

Q. Callaway Plant Evacuation Time Estimate report, June 2012; Callaway Plant Unit 1,

Union Electric Co. Facility Operating License NPF-30; Thomas Elwood (314) 225 -

1905;http://pbadupws.nrc.gov/docs/ML1220/ML12200A026.pdf

R. Nine Mile Point Development of Evacuation Time Estimates; November 2012,

Prepared for Constellation Energy and Entegry; Prepared by KLD Engineering, P.C

43 Corporate Drive, Hauppauge, NY 11788; mailto:[email protected]

S. Hurricane Isabel Assessment, March 2005; NOAA Coastal Services Center,

Hurricane Planning and Impact; https://www.csc.noaa.gov/hes. Prepared for USACE,

Philadelphia and Wilmington District and Federal Emergency Management Agency

(FEMA) Region III & IV; Prepared by Post, Buckley, Schuh and Jeringan, Inc. 1901

Commonwealth Lane, Tallahassee, Fl. 32303.

T. Hurricane Hugo Assessment, January 2000; NOAA Coastal Services Center,


South Atlantic District and Federal Emergency Management Agency (FEMA) Region

http://www.ce.udel.edu/UTC/Dalton_Summer08

http://www.census.gov/2010census/

http://pbadupws.nrc.gov/docs/ML0731/ML073110546.

50

IV; Prepared by Post, Buckley, Schuh and Jeringan, Inc. 134 South Bronough St.

Tallahassee, Fl. 32301.

U. Hurricane Opal Assessment, October 1995; NOAA Coastal Services Center,


Mobile District and Federal Emergency Management Agency (FEMA) Region IV;

Prepared by USACE, Philadelphia District, September 1996.

V. Hurricane Floyd Assessment, May 2000; NOAA Coastal Services Center, Hurricane

Planning and Impact; https://www.csc.noaa.gov/hes. Prepared for USACE, Savanna

District and Federal Emergency Management Agency (FEMA) Region IV; Prepared

by Post, Buckley, Schuh and Jeringan, Inc. 1901 Commonwealth Lane, St.

Tallahassee, Fl. 32303.

W. Hurricane Andrew Assessment, January 1993; NOAA Coastal Services Center,


South Atlantic Division and Federal Emergency Management Agency (FEMA)

Region III & IV; Prepared by Post, Buckley, Schuh and Jeringan, Inc. 134 South

Bronough St. Tallahassee, Fl. 32301.

X. Framework for Modeling Emergency Evacuation, University of Central Florida, 2005

Y. A Case Study of Hurricane Evacuations of Galveston Island

Z. Hooks, Elise-Miller, and Tarnoff, Phil. “Traffic Signal Timing for Urban Evacuation:

Draft Final Report”, Maryland Center for Advanced Transportation Technology,

University of Maryland for the Federal Highway Administration. 18 August 2005

AA. Chen, M., L. Chen and E. Miller-Hooks (2007). “Traffic Signal Timing for Urban

Evacuation, ”Special Emergency Transportation Issue of the ASCE Journal of Urban

Planning and Development

51

APPENDIX A

TEST PLAN

M#s MEASURE METRIC DATA

ELEMENT

MEASUREMENT

METHOD DATA ANALYSIS

Does RtePM consider evacuation times and routes for different types of disasters? M1-M11.12

1. Man-made

disasters

M1 a. Chemical Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s/sharehol

ders, these

measures will

be evaluated as

satisfactory

i. Dispersion

of lethal

clouds due

to wind

speed and

direction

Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s/sharehol

ders, these

measures will

be evaluated as

12

RtePM is an evacuation model used to calculate evacuation times for any type of disaster. It is designed for the user to select a specific area to evacuate, no matter what the disaster may be. The model is designed to be used anywhere that road and population data can be obtained

52


ELEMENT

MEASUREMENT


satisfactory

ii. Persistency

of lethal

clouds

based upon

dispersion

and time

Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s

/shareholders,

these measures

will be

evaluated as

satisfactory

M2 b. Nuclear

(terrorist

attack)

Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s/sharehol

ders, these

measures will

be evaluated as

satisfactory

M3 c. Conventional

Explosive

Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

53


ELEMENT

MEASUREMENT


and collect

surveys from

SME’s.

SME’s/sharehol

ders, these

measures will

be evaluated as

satisfactory

M4 d. Fire Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s/sharehol

ders, these

measures will

be evaluated as

satisfactory

2. Natural Disasters

M5 a. Hurricane Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s/sharehol

ders, these

measures will

be evaluated as

satisfactory

M6 b. Earthquake Y/N Test director

observation

Run RtePM

scenarios,

observe

If Test director

observes

positive results

54


ELEMENT

MEASUREMENT


evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

from model

runs, and gains

positive

feedback from

SME’s/sharehol

ders, these

measures will

be evaluated as

satisfactory

M7 c. Flooding Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s/sharehol

ders, these

measures will

be evaluated as

satisfactory

M8 d. Fires Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s/sharehol

ders, these

measures will

be evaluated as

satisfactory

55


ELEMENT

MEASUREMENT


3. Catastrophic

accidents

M9 a. Aircraft

crashes

Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s/sharehol

ders, these

measures will

be evaluated as

satisfactory

M10 b. Train wrecks

/derailments

Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

surveys from

SME’s.

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s/sharehol

ders, these

measures will

be evaluated as

satisfactory

M11 4. Industrial

accidents

(Factory or

Vehicular)

Y/N Test director

observation

Run RtePM

scenarios,

observe

evacuation time

results, conduct

interviews from

SME’s, distribute

and collect

If Test director

observes

positive results

from model

runs, and gains

positive

feedback from

SME’s/sharehol

56


ELEMENT

MEASUREMENT


surveys from

SME’s.

ders, these

measures will

be evaluated as

satisfactory

Does RtePM utilize the most current population estimates and is population database

adjustable? M12-M17

M12 1. Source of

population data

Y/N-

within 5%

accuracy

1. US Census

data from

2010.

2. RtePM

Total

Population

data

Sample small,

medium and

large population

areas to

compare

population

values of RtePM

with that of

2010 Census

If Test director

observes

positive results

from

population

comparisons,

these measures

will be

evaluated as

satisfactory

M13 2. Adjustable

density levels

Y/N-100%

accurate

1. Population

change

2. Total

population

3. Total

Vehicles

needed to

evacuate

Test for accurate

change in

population

density by

adjusting the

population

change block (%)

by 10-25% in

order to

determine that

these changes

affect total

population and

total vehicles

results in RtePM.

If Test director

observes

positive results

from

population

density

comparisons,

these measures

will be

evaluated as

satisfactory

57


ELEMENT

MEASUREMENT


M14 3. Seasonal

Scalability

Y/N-100%

accurate

Seasonal

population

increase and

total vehicles

required to

evacuate

Validation of

populations

when adding or

subtracting for

seasonal

variations will be

investigated for

correctness.

If Test director

observes

positive results

from

population

seasonal

scalability

comparisons,

these measures

will be

evaluated as

satisfactory

M15 4. Number of

evacuees per

vehicle

Y/N-100%

accurate

People/Vehic

le block and

total vehicle

Test for accurate

change in total

number of

vehicles by

adjusting the

people/vehicle

block (1-4).

If Test director

observes

positive results

from the total

vehicle

comparison

test, these

measures will

be evaluated as

satisfactory

M16 5. Shelters/Evacuat

ion Centers

Y/N Viewable

shelters

Add/Subtract

shelters in

various

scenarios

If Test director

observes

positive results

from shelter

manipulation

comparisons,

these measures

will be

evaluated as

satisfactory

M17 a. Capacity Y/N Shelter Change capacity

58


ELEMENT

MEASUREMENT


Capacity of shelter

Does RtePM model the most current and accurate Roads and Routes? M18-M25

M18 1. Must have

logical evacuee

flow

Y/N 1. End Points Reconfigure end

points, location

of end points,

number of

endpoints

If Test director

observes

positive results

for a logical

traffic flow,

these measures

will be

evaluated as

satisfactory

M19 2. Number of lanes

available

Y/N 1. RtePM

2. Google

Maps-Street

View

Go to Websites

(Google) and

compare with

RtePM # of lanes

data.

If Test director

observes

positive results

after changing

number of lanes

available, these

measures will

be evaluated as

satisfactory

M20 3. Use of reversible

lanes

Y/N Add use of

reversible

lanes

Build scenarios

with reversible

lanes and

calculate

evacuation

times for

different

scenarios

If Test director

observes

positive results

with the use of

reversible lanes,

these measures

will be

evaluated as

satisfactory

M21 4. Use of shoulder

lane

Y/N Add use of

shoulder

traffic

Include use of

shoulder in

different

If Test director

observes

positive results

59


ELEMENT

MEASUREMENT


scenarios and

determine if

evacuation time

changes

accordingly

from use of a

shoulder lanes,

these measures

will be

evaluated as

satisfactory

M22 5. Road closures Y/N Close roads Close roads

during

evacuation

scenario

construction and

determine if

evacuation

times differ

If Test director

observes

positive results

from the closing

of roads, these

measures will

be evaluated as

satisfactory

a. Tunnels

blocked

Y/N Block tunnels

b. Roads blocked Y/N Block roads

c. Bridges

blocked

Y/N Block bridges

M23 6. Road speed limit Y/N 1. RtePM

2. Google

Maps-Street

View

Go to Websites

(Google) and

compare with

RtePM Road

Speed Limit

data.

If Test director

observes

positive results

when speed

limit changes

are made, these

measures will

be evaluated as

satisfactory

M24 7. Identify

bottlenecks

Y/N Insert

bottlenecks

Identify known

bottlenecks/brid

ges, tunnels,

merges, make

changes to plan

If Test director

observes

positive results

when

considering

60


ELEMENT

MEASUREMENT


and determine

calculation

changes

bottlenecks,

these measures

will be

evaluated as

satisfactory

a. Bridges Y/N

b. Tunnels Y/N

c. Merges Y/N

M25 8. Alternate routes Y/N Change

endpoints

Add/Delete

endpoints and

observe change

in Time to

evacuate and

direction of

evacuation.

If Test director

observes

positive results

from change in

endpoint, this

measures will

be evaluated as

satisfactory

Does RtePM model make sense to the user when modeling the effects of traffic M26-

M29

M26 1. Traffic Density Y/N None

Low

Medium

High

Run scenarios

with different

levels of traffic

density and

observe change

in evacuation

time.

If Test director

observes

positive results

when traffic

density

variables are

changed, these

measures will

be evaluated as

satisfactory

a. Congestion Y/N

i. Number of Y/N

61


ELEMENT

MEASUREMENT


vehicles

ii. Seasonal

tourists

Y/N

b. Accidents Y/N

c. Bridges

blocked

Y/N

d. Tunnels

blocked

Y/N

e. Roads blocked Y/N

f. Railways

blocked

Y/N

M27 2. Time of Day Y/N Nighttime

(Census)

Daytime/Wor

kweek

(Land Scan)

Run scenarios

with different

time of day and

observe change

in evacuation

time.

If Test director

observes

positive results

when time of

day variables

are entered,

these measures

will be

evaluated as

satisfactory

M28 3. Realistic effects

of congestion on

travel routes,

speeds and trip

lengths

Y/N Increase/Dec

rease

congestion

levels

Run scenarios

with different

congestion

levels and

observe change

in evacuation

time, speed.

If Test director

observes

positive results

from time of

evacuation due

to traffic

congestion,

these measures

will be

evaluated as

satisfactory

62


ELEMENT

MEASUREMENT


M29 4. Fuel Availability Y/N N/A N/A

Does RtePM accurately determine Time to Evacuate M30-M33

M30 1. When

evacuation starts

Y/N Evacuation

Start time

Observe change

in evacuation

time when

changing

evacuation start

time.

If Test director

observes

positive results

when time of

evacuation is

changed, these

measures will

be evaluated as

satisfactory

M31 2. Staggered

evacuations

Y/N Stagger

evacuation

start time

Observe change

in evacuation

time when

staggering

evacuations.

If Test director

observes

positive results

when using

staggered

evacuations,

these measures

will be

evaluated as

satisfactory

M32 3. Evacuation zones Y/N Add/Delete

evacuation

zones

Observe change

in evacuation

time when

adding or

deleting

evacuation zone.

If Test director

observes

positive results

when

evacuation

zones are

modified, these

measures will

be evaluated as

satisfactory

M33 4. Advance notice Y/N N/A If Test director

63


ELEMENT

MEASUREMENT


observes

positive results

when advanced

notice is given,

these measures

will be

evaluated as

satisfactory

Does RtePM provide visual displays that are user friendly M34-M35

M34 1. Provide visual

displays

If Test director

observes

positive results

with reference

to visual

displays, these

measures will

be evaluated as

satisfactory

a. Locations of

incidents in

familiar terms

(i.e., street

addresses vice

grid

coordinates or

lat/long)

Y/N

b. Global maps

capable of

zooming to a

building in a

city

Y/N

c. Operator

friendly

Y/N

64


ELEMENT

MEASUREMENT


graphical user

interface (GUI)

d. Display terrain

features

Y/N

M35 2. Rapid Use Case

generation

capability

Y/N If Test director

observes

positive results

for rapid

scenario

generation,

these measures

will be

evaluated as

satisfactory

a. Ability to easily

export and

modify Use

Case data from

one Use Case

to another

Y/N

b. Ability to

import real

data from

databases

Y/N

3. Ability to save a

Use Case

Y/N

Does RtePM consider the impact of weather on evacuation time M36

M36 1. Does the

evacuation time

change with

Y/N Slow traffic with

higher density,

lower speeds,

If Test director

observes

positive results

65


ELEMENT

MEASUREMENT


changes in the

weather?13

and closed

roads.

from evacuation

times, these

measures will

be evaluated as

satisfactory

13

RtePM is not designed to account for changes in the weather

66

APPENDIX B

CRITICAL FEATURES AND ESSENTIAL ELEMENTS

DDL Omni Engineering identified eight critical features and the underlying essential elements of

an effective evacuation model, before operating RtePM. After gaining access to RtePM,

observations were made and identified in the right hand column.

CF

&

EE #’s

CRITICAL FEATURES AND

ESSENTIAL ELEMENTS COMMENTS

The model shall represent, with acceptable accuracy, the effects of the following Critical Factor

and Essential Elements:

CF1000 I. Different types of Disasters RtePM is an evacuation tool used for

the types of disasters listed below. It

is designed to select a specific area to

evacuate, no matter what the disaster

may be.

EE1001 1. Man-made disasters

EE1002 a. Chemical

EE1003 i. Dispersion of lethal clouds

due to wind speed and

direction

Weather factors need to be included

in order to model dispersion

direction. Specifically wind direction

and speed.

EE1004 ii. Persistency of lethal clouds

based upon dispersion and

time


in order to model dispersion

direction.

EE1005 b. Nuclear (terrorist attack) Easy to identify evacuation zone and

time to evacuate

EE1006 c. Conventional Explosive Easy to identify evacuation zone and

time to evacuate

EE1007 d. Fire Easy to identify evacuation zone and

time to evacuate

EE1008 2. Natural Disasters

EE1090 a. Hurricane Functionality of model provides for

multi-day evacuation which is needed

for hurricane evacuation

EE1010 b. Earthquake Easy to identify evacuation zone and

time to evacuate

EE1011 c. Flooding Does not display terrain features that

adequately identify potential flood

zones that would affect road closures.

67

CF

&

EE #’s



EE1012 d. Fires Easy to identify evacuation zone and

time to evacuate

EE1013 3. Catastrophic accidents

EE1014 a. Aircraft crashes Easy to identify evacuation zone and

time to evacuate

EE1015 b. Train wrecks/derailments Easy to identify evacuation zone and

time to evacuate

EE1016 4. Industrial accidents (Factory or

Vehicular)

EE1017 a. Chemical

EE1018 i. Dispersion of lethal clouds

due to wind speed and

direction


in order to model dispersion direction.

Specifically wind direction and speed.

EE1019 ii. Persistency of lethal clouds

based upon dispersion and

time


in order to model dispersion direction.

Specifically wind direction and speed.



CF2000 II. Must accurately model Populations:

EE2001 1. Source of population data Population data has been obtained

through the 2010 census.

EE2002 2. Adjustable density levels Under the POPULATION BLOCKS

TAB, the following is viewable, and

is adjustable:

1. Census Block ID Group

2. Daytime population

3. Nighttime Population

4. Average household size

5. Number of Households

Under the CONFIGURATION TAB,

Population Change (%) is adjustable.

This tab is used to adjust the percent

that the population has increased or

decreased since the population data

was obtained. Can range from 0-

100% and defaults to 0%.

EE2008 3. Seasonal Scalability Under the SEASONAL TAB,

seasonal populations can be adjusted

and impact evacuation times will

68

CF

&

EE #’s



adjust. The seasonal populations are

illustrated on the map display by a

grid overlay that may be turned on or

off using the “Show Layer” select

box. Seasonal populations are defined

as surge type populations that fall

outside of the census data. Scenarios

may be run with seasonal populations

active or not active to aid planners in

understanding variations in

evacuation events. Clicking the

“Add” button defines seasonal

populations:

EE2009 4. Number of evacuees per vehicle The number of passengers (on

average) that will be in each

evacuating vehicle is adjustable under

the configuration TAB. Maximum

number of passengers per vehicle is

4. This number is adjustable.

EE2010 5. Shelters/Evacuation Centers Shelters may be added to the

evacuation area using the “Add”

icon. They will be displayed on the

map as a small green house icon. If

shelters exist in the evacuation zone

already they may be selected using

the Polygon or bounding box tools.

Shelters are also removable.

Selecting the shelter to be removed

by choosing the “Remove” tool on

the “Shelters” tab and selecting them

with the mouse on the map display

do this. The checkbox making a

shelter active may also be unchecked

allowing the configuration of the

shelter to remain, but not factored

into the current simulation.

Configurable shelter location, should

be automated.

EE2011 a. Capacity Shelter capacity is adjustable

69

CF

&

EE #’s





CF3000 III. Must accurately model Roads and

Routes:

EE3001 1. Must have logical evacuee flow Under the Evacuation Zone sub tab

is End Point Assignments.

Evacuation End Points define where

evacuees are being sent. End Points

are not available until after Roads are

assigned. Endpoints are displayed as

yellow circles on the map. Each

yellow circle corresponds to one of

the end points named in the table.

Mousing over the end points in the

table will highlight their location on

the map.

EE3002 2. Number of lanes available There are also four filters that are

applicable to road networks. They

are “Highway,” “Major Arterial,”

“Minor Arterial,” and “Smaller.”

“Highway” defines the routes of

egress as Highways, and the others

follow suit. “Smaller” refers to non-

major roadways included in the

evacuation area. Checking or

unchecking the filter boxes will

include or preclude the roadway type

from the evacuation. Filtering of

specific roadways may also be done

using the button to select a specific

roadway for editing.

EE3003 3. Use of reversible lanes By clicking on the tab titled “Roads,”

then “Modified Roads” and then

click on the “Add A Modification

Section” button (green plus sign) in

the upper right hand corner. This will

create a section row.

Click the “Edit” button under the

column “Segments.”

On the map, starting at the end point,

70

CF

&

EE #’s



click on the segments of roadway

back to the point at which you want

to start the contraflow operation. As

you hoover over the roadway

segment it will turn green, it will turn

purple after you click on it.

In the tool, click on the “Editing”

button.

Check the box under the column

“Contraflow”

In the open box under “Contraflow”

indicate the numbers of lanes on this

side of the highway that should not

be contra flowed, for our example,

“1.”

Click on the tab “Additional Roads”

and then click on the “Add an

Additional Section” button (green

plus sign) in the upper right hand

corner. This will create an additional

road row.

Click the “Choose” button under the

column “Points”

Go back to the starting point of

contraflow on the map and link the

contraflow segment with the opposite

of the roadway. This will create a lane

to move cars directionally into the

lane(s) to be contra flowed.

You can choose the free flow speed

of this new segment as well as specify

how many of the total lanes in the

new segment will be used.”

Click on the “Choosing” button, it

will return to “Choose,” completing

the operation

The reversible lane function needs to

be modified for ease of operation.

Sometimes a road needs to be added,

and other times a road does not need

71

CF

&

EE #’s



to be added to complete the contra-

flow. It is not always clear if a

contra-flow has been added, and if

there is an adjustment to evacuation

times.

EE3004 4. Use of shoulder lane Modified in the same section as

reversible lanes

EE3005 5. Road closures Modified in the same section as

reversible lanes. Tunnels, roads and

bridges are easily opened and closed.

EE3006 a. Tunnels blocked Modified under the “ROADS” tab

EE3007 b. Roads blocked Modified under the “ROADS” tab

EE3008 c. Bridges blocked Modified under the “ROADS” tab

EE3009 6. Road speed limit Current speed limits are integrated,

and can be changed. Modified under

the “ROADS” tab

EE3010 7. Identify bottlenecks Bottlenecks are identified during

playback in the throughput mode.

Bottlenecks are not easily identified

during playback unless playback is

paused

EE3011 a. Bridges Choke points identified during

playback in the throughput mode

EE3012 b. Tunnels Choke points identified during


EE3013 c. Merges Choke points identified during


EE3014 8. Alternate routes Alternate routes can be manipulated

by changing location of end-points

and/or closing of roads



CF4000 IV. Must model the effects of traffic

EE4001 1. Traffic Density

EE4002 a. Congestion Choke points are identified in

playback mode

EE 4003 i. Number of vehicles Controlled by population and

number of passengers per vehicle.

Other options available such as

number of vehicles being towed.

72

CF

&

EE #’s



EE4004 ii. Seasonal tourists Allows seasonal populations to be

considered during evacuations. The

seasonal populations are illustrated

on the map display by a grid overlay

that may be turned on or off using

the “Show Layer” select box.

EE4005 b. Accidents Accidents can be simulated in the

roads section by closing a road or

changing the number of lanes

available.

EE4006 c. Bridges blocked Blocked bridges can be simulated in

the roads section by closing a road or


available.

EE4007 d. Tunnels blocked Blocked tunnels can be simulated in

the roads section by closing a road or


available.

EE4008 e. Roads blocked Blocked roads can be simulated in

the roads section

EE409 f. Railways blocked RtePM does not account for mass

transit. In the case of a train

blocking a road, use roads section to

close a road.

EE4010 2. Time of Day Daytime/Nighttime census data is

considered

EE4011 3. Realistic effects of congestion on

travel routes, speeds and trip lengths

Evacuation times vary accordingly

with changes in traffic congestion.

EE4012 4. Fuel Availability Fuel availability functions are not

available



CF5000 V. Must accurately determine Time to

Evacuate

EE5001 1. When evacuation starts Ability to set evacuation start time.

EE5002 2. Staggered evacuations RtePM gives you the ability to create

phased evacuations. The user must

first edit or create a new scenario as

detailed earlier in this users guide.

Then the user should select the

“Evacuation Area” tab.

73

CF

&

EE #’s



EE5003 3. Evacuation zones This allows the user to create

evacuation zones that can have

delayed start times and have different

response and behavioral parameters

EE5004 4. Advance notice Evacuations can be created using

RtePM as a planning tool with

advanced notice



CF6000 VI. Must accurately model the effects of

Emergency Services

RtePM does not a requirement to

model the effects of Emergency

Services

EE6001 1. Traffic Police Does not include location or effects

of emergency services.

EE6007 2. Ambulance Does not include location or effects


EE6012 3. Fire Trucks Does not include location or effects


EE6017 4. Tow Trucks Does not include location or effects




CF7000 VII. Model/System capabilities shall include:

EE7001 1. Provide visual displays

EE7002 a. Locations of incidents in familiar

terms (i.e., street addresses vice

grid coordinates or lat/long)

Locations of incidents may be

identified with polygon or rectangle

feature

EE7003 b. Global maps capable of zooming

to a building in a city

Provides the ability to view a map in

terms of a street view, an aerial view,

or a topographic view

EE7004 c. Operator friendly graphical user

interface (GUI)

1. Contra flow difficult

2. Need hour glass when calculating

3. Evacuation planning wizard

interface would be useful down the

left hand side to show user progress

of planning while map is still visible

4. User guide is in development,

needs updating. Help window with

user guide embedded in tool may be

useful.

EE7005 d. Display terrain features Terrain features that would identify

potential flood zones needs to be

74

CF

&

EE #’s



more robust Flood zone overlay may

be useful in planning for evacuation.

Topo option is available on the map,

but needs to link to flood database

EE7006 2. Rapid Use Case generation capability

EE7007 a. Ability to easily export and

modify Use Case data from one

Use Case to another

Scenario can be saved to personal

computer. If someone deletes

scenario from RtePM, user can

import that scenario so it allows user

to secure scenarios in the current

version of RtePM. It also provides an

easy way to move scenarios from one

server of RtePM to another.



CF8000 VIII Impact of weather on evacuation RtePM does not have a requirement

to include the effects of weather

EE8001 1. Models affected by Weather Workarounds such as speed limits,

road/bridge closures can simulate the

effects of inclement weather.

EE8002 a. Weather is modifiable. Road closure for high winds or heavy

seas on bridges, or flooding for

tunnels

EE8003 b. Day / Night. Model effects of

Day / Night

Day/Night taken into consideration

75

APPENDIX C

RtePM ENHANCEMENTS

VMASC is currently in the process of enhancing the existing RtePM tool by the addition of

evacuation time improvements and widget/overlay additions. By combining the listed features

below, a knowledgeable local emergency manager can fine tune their evacuation time estimates

considerably, making these new features valuable.

A. Evacuation Time Improvements include:

1. Probabilistic techniques that enhance the accuracy of RtePM. The prototype RtePM

is highly deterministic, meaning the tool’s output is directly dependent on the inputs

provided by the user (response time, participation rate, route availability, etc.). The

existing tool is thus very useful for providing a rough estimate of evacuation times

and for assessing the impact of discrete changes to entered characteristics, but limited

in its ability to provide the range of results for which planners should prepare. The

natural advancement of the tool would include addition of probabilistic measures.

Early indications of DDL Omni Engineering beta testing reveal that this feature is

functional, and will offer valuable data to the emergency manager, but further

validation is required.

2. Vehicular accidents and incidents interface that will significantly increase the

accuracy of evacuation times, especially when combined with the probabilistic

calculations. Early testing indicates that the feature is working and impacts the time

to evacuate in a densely populated area. It does not impact evacuation times in rural

areas since traffic density is not a major factor in calculating evacuation times.

3. Public transit interface allows the user to enter the percentage of the total population

that may be evacuating by public bus and rail. It does not model evacuation time for

public bus or rail, it simply removes that amount (%) of population from the number

of people evacuating by private vehicle. Testing proves that the higher percentage of

population using public transportation results in decrease of evacuation time due to

the decrease in vehicles on the roadway.

4. Pedestrian interface allows the user to enter the percentage of the total population that

may be evacuating by foot. It does not model evacuation time for pedestrians, it

simply removes that amount (%) of population from the number of people evacuating

by private vehicle. Testing proves that the higher percentage of population evacuating

by foot results in decrease of evacuation time due to the decrease in vehicles on the

roadway.

76

B. WIDGET/Overlay additions

The following widgets were added to the prototype RtePM. The intention was to add

capabilities that provide flexibility when modeling plume dispersal or displaying KML

files/shape file overlays. DDL Omni Engineering did not validate widget additions 1-4

listed below. But we did ensure that the additional features, when selected, displayed on the

RtePM map. These four additional widgets displayed properly and are an effective

enhancement to RtePM.

1. ALOHA Threat Zone - ALOHA (Areal Locations of Hazardous Atmospheres) is a

computer program developed by Office of Emergency Management, Environmental

Protection Agency and Emergency Response Division, National Oceanic and

Atmospheric Administration. It is designed to model chemical releases for emergency

responders and planners. It can estimate how a toxic cloud might disperse after a

chemical release - as well as several fires and explosions scenarios. It incorporates

source strength, as well as Gaussian and heavy gas dispersion models and an

extensive chemical property library. Model graphical output includes a "footprint"

plot of the area downwind of a release, where concentrations may exceed a user-set

threshold level. Within RtePM, ALOHA acts as a graphical interface between the

user's computer and the ALOHA program running on the RtePM server and provides

a subset of the functionality of the standalone program.

The following are actions available to the user:

• Select/Load a base script file.

• Change any configuration parameters via the Configure Script widget icon

• Select the script filename and output filename to store on the server.

• Generate the Aloha output file.

• Save a copy of the script and output files on the user's computer (optional).

• Display Aloha output files using the Display Plume Widget icon

• Remove loaded Aloha files.

• Override stylistic parameters of displayed Aloha files.

• Change visibility and opacity of displayed Aloha files.

• Threat Zone - allows user to draw threat zone and point of origin

2. HotSpot Plume - allows user to generate a plume. Plume-HotSpot was developed by

Lawrence Livermore National Laboratory's (LLNL) National. It provides a first-

order approximation of the radiation effects associated with the atmospheric release

of radioactive materials. The HotSpot program was created to equip emergency

response personnel and planners with a fast, field-portable set of software tools for

evaluating incidents involving radioactive material. The software is also used for

safety-analysis of facilities handling radioactive material. This program is designed

77

for short-range (less than 10 km), and short-term (less than a few hours) predictions.

Within RtePM, PLUME acts as a graphical interface between the user's computer and

the PLUME program running on the RtePM server.


• Select/Load the base configuration file

• Select a point of origin via the Select Point of Origin icon

• Change any configuration parameters via the Change Configuration Parameters

• Select the configuration filename and output filename to store on the server

• Generate the HotSpot plume

• Save a copy of the configuration and output files on the user's computer

(optional)

• Display the HotSpot output plume using the Display Plume Widget icon

• Remove loaded HotSpot files.

• Override stylistic parameters of displayed HotSpot files.

• Change visibility and opacity of displayed HotSpot files.

3. KML (Keyhole Markup Language) Layer allows for the loading and display of KML

Overlays on the RtePM map.


• KML files can be loaded from the user's computer or the server, but not KMZ

files (compressed KML files with optional sub-directories and files).

• KML or KMZ files can be loaded from publically accessible sites.

• Consideration should be taken as to the order of loading overlays so that shapes

of interest are not obscured by other shapes.

• The following are actions available to the user:

• Select/Load KML files.

• Remove loaded KML files.

• Override stylistic parameters of displayed KML files.

• Change visibility and opacity of displayed KML files.

4. Load Shape Files - allows for the loading and display of shape file overlays on the

RtePM map.


• Select/Load shape files.

• Remove loaded shape files.

• Set stylistic parameters for display of subsequent shape files loaded.

• Change visibility and opacity of displayed shape files.

78

Testing has been performed on the enhanced features listed above, but at this time, many more

data sets will be required before a proper validation can be performed. DDL Omni Engineering

has every confidence that, in the future, the enhanced version of RtePM will provide evacuation

planners with an even more effective decision support system that allows multiple scenarios to

be evaluated.

Documents

INDEPENDENT VERIFICATION AND VALIDATION (IV&V) OF REAL ...rtepm.vmasc.odu.edu/Appendix_D_IVV.pdf · The Commonwealth of Virginia and the Virginia Department of Emergency Management