Mike Geller Department of Systems Engineering and Engineering Management Charles V. Schaefer Jr. School of Engineering Stevens Institute of Technology Hoboken, New Jersey 07030 mgeller@stevens.eduIntroduction to Modeling and Simulation

Lesson ObjectivesLesson Objectives Introduction to Simulation Introduction to Process Generators ExamplesNext Lessons Queuing Theory More Examples Simulation in Practice

The beginning isthe most importantpart of the work(Plato, 400 B.C.)Getting Started

Studying Complex Systems*

A model is a physical, mathematical or logical representation of a system, entity, phenomenon, or process. There is no movement in a model.What is a Model?Example: a plastic replica of a car, or a mathematical equation that predicts the probability of an event occurringDOD Modeling and Simulation Information Analysis Center (MSIAC) *

Modeling is the application of a standard, rigorous, structured methodology to create and validate a physical, mathematical, or otherwise logical representation of a system, entity, phenomenon, or process.What is Modeling?DOD Modeling and Simulation Information Analysis Center (MSIAC) *

Classification of ModelsPhysicalMathematicalConceptual*

Physical Model. A model whose physical characteristics resemble the physical characteristics of the system being modeled.What is a Physical Model?DOD Modeling and Simulation Information Analysis Center (MSIAC) Example: a wooden replica of an airplane *

Conceptual Model. A statement of the content and internal representations which are the users and developers combined concept of the model. It includes logic and algorithms and explicitly recognizes assumptions and limitations.What is a Conceptual Model?DOD Modeling and Simulation Information Analysis Center (MSIAC) *

Mathematical Model. A symbolic model whose properties are expressed in mathematical symbols and relationships.What is a Mathematical Model?DOD Modeling and Simulation Information Analysis Center (MSIAC) Example: a model of a nation's economyexpressed as a set of equations*

Classification of ModelsPhysicalMathematicalStaticDynamicConceptual*

A model of a system in which there is no change.

Static ModelsExample: a scale model of a house studied for its appearanceDOD Modeling and Simulation Information Analysis Center (MSIAC) *

A model of a system in which there is change, such as the occurrence of events over time or the movement of objects through space. Dynamic ModelsExample: a model of a bridge that is subjected to a moving load to determine characteristics of the bridge under changing stressDOD Modeling and Simulation Information Analysis Center (MSIAC) *

Dynamic Modelsf(t)InputsOutput(s)VariablesConstants*Over Time

Classification of ModelsMathematicalStaticDynamicContinuousDiscrete*

A mathematical or computational model whose output variables change in a continuous manner. Continuous ModelsExample: a model a model depicting the rate of air flow over an airplane wingIEEE Standard Glossary of Modeling and Simulation Std 610.3-1989*

SumMultiplyDivideIntegratorX1(t)X2(t)X3(t)X4(t)y(t)EXAMPLEXContinuous Models*

A mathematical or computational model whose output variables take on only discrete values.Discrete ModelsExample: a model that predicts an organizations inventory levels based on varying shipments and receiptsIEEE Standard Glossary of Modeling and Simulation Std 610.3-1989*

What is a Simulator?A device, computer program, or system that performs simulation.IEEE Standard Glossary of Modeling and Simulation Std 610.3-1989*

Simulation is the implementation of a model over time. It shows how the model works. It is a technique used for testing, analysis, or training, where a model can represent real world systems or concepts. A simulation moves. You can see the model(s) in the simulation movingwhether it shows military units moving across a battlefield or engine parts moving in a simulated car engine. DOD Modeling and Simulation Information Analysis Center (MSIAC) What is Simulation?*

SIMULATION: a technique that imitates the operation of a real-world system as it evolves over time. Results obtained are SAMPLE observations of sample STATISTICS. The process of designing a mathematical or logical model of a real-system and then conducting computer-based experiments with the model to describe, explain, and predict the behavior of the real system

STOCHASTIC SIMULATION MODEL: a model that contains one or more random variables represented by probability distributions.

MONTE CARLO SAMPLING: a mathematical technique for selecting numbers randomly according to a probability distribution for use in a trial runDefinitionsFrom the Latin word simulatus, past participle of simulare to copy, represent, feign, from similis like

EVENT DRIVEN SIMULATIONAn event is an occurrence in a system at which changes to a system occur.

Events take place at an instance of time.

A system is modeled by defining the events that occur in the system and describe what takes place at certain times.DefinitionsSimulation is the only way to represent (model, test, design, etc.) complex systems

Modeling Processes with Events and ActivitiesCustomerArrivalStart ofServiceEnd ofServiceCustomerDeparture(Event)(Event)(Event)(Event)A process is a model with logic (i.e., if, then, else)WaitingServiceService(Activity)(Activity)(Activity)An events behavior is described by some mathematical function/distribution these are often called process generatorsSimpleQueuingModel

PRIMARY OBJECTIVE OF STOCHASTIC SIMULATION: To reproduce realistically the behavior of the system being studied, including the variability of the random variable(s) included in the system.

STOCHASTIC SIMULATION MODEL:A model that contains one or more random variables.

MONTE CARLO SIMULATION: A representation of a system at a particular point in time. A STATIC simulation model (i.e., each day is an independent simulation; not effected by events that occurred previously)More Miscellaneous Information

More Miscellaneous InformationStochastic Simulation for Decision Analysis: Monte Carlo Simulation Discrete Event Driven SimulationNot Interested at the Moment: Deterministic Simulation (FD, FEM, 3D)

Need various types modelsAdvances in system development ultimately rely on well-constructed predictive models

Applications:traditional fields such as electrical and mechanical engineering newer domains such as information and bio-technologies

Using appropriate simulation software, we can derive solutions to difficult problems using such models

Success often depends on having a variety of modeling approaches available to formulate the right model for the particular issue at hand

Therefore, a broad familiarity with different types of models is desirable *

System Modeling and SimulationSystemInputsOutputsf(t)Data~VariablestimeModelGeneratorAnalysisHistoricalEstimatedHypothesisStochasticRandomDecisionExperimentHypothesis TestingEntertainment*

Physical Models Conceptual Models Mathematical Models Static Models Dynamic Models Continuous Systems M&SStochastic M&SContinuous M&SSystems Dynamics M&S Discrete-event Systems M&SStatistical M&SMonte Carlo simulation

Types of Modeling & Simulation (M&S)

*

ComparisonsDiscrete Event Dynamic simulation (time sensitive) Event driven Using attributes, unique characteristics are assigned to items which can then tracked throughout the model Blocking, balking, reneging easily handled Variability (associated with time) captured easily Manufacturing, service, business processes, strategic thinking, networks, systems engineering, etc. Animation is used great verification/validation toolMonte Carlo Static simulation not time axis Random numbers must be repeated for each query/junction Event driven is difficult Blocking, balking, and reneging is very difficult Difficult to capture variability as a function of time

Advantages of SimulationThe advantages of simulation are:1) Once the model is explained, most people can understand it and accept its results as legitimate representations of the system under consideration (a simulation is more "intuitive"),2) Simulation can be used for complex, real-world situations or conditions that are not included in analytical models,3) We can simulate extended periods of time in a short period of time on a computer,4) It is much less expensive to build something in a computer language and experiment with the model than it is to construct the physical system for experimentation,

Advantages of SimulationThe advantages of simulation are:5) Simulation allows for easier "what-if" analysis and variations on the existing model (sensitivity analysis),6) Relatively straight forward; minimum cost,7) Easier to apply than analytical methods,8) Greater flexibility in representing the real system - fewer simplifying assumptions,9) Precludes loss of people lives & damage to the environment, and10) Model can be used repeatedly.

Disadvantages of Simulation1) Is not an optimizer, 2) Does not lead to fundamental understanding (we observe outcomes on a process, but may not understand why the outcomes are as they are),3) An abused analytical tool that is often used in lieu of physical models,4) The best simulation languages (most complex) and models can be expensive and require a great investment in time to learn the simulation program,5) Simulation models do not provide optimal solutions.6) Only the conditions that are included in the model can be examined, and7) You may not discover fundamental relationships that are sometimes illuminated by analytical models.

Areas of Simulation ApplicationQueuing SystemsInventory Control (demand is rarely known w/certainty)Production and Manufacturing (work scheduling, assembly line balancing)Public Service OperationsEnvironmental and Resource Analysis (EIS, Energy Utilization, pollution)Military Applications (weapons systems, wargames)

Some Examples (From Experience)MJG ExamplesF14A A/C Maintenance SimulationOcean Systems SimulatorFDNY Environmental SimulatorFDNY Dispatch System (Computer) Performance SimulatorTelecommunication SimulationsNetwork Design & PerformanceRF Environment SimulationsAtmospheric Simulation for LaserComSatellite Processing PerformanceMilitary Systems PerformanceFlight SimulatorsWeapon System SimulatorsWar Game Simulations

Simulation Modeling ProcessDefine the System and Identify the Problem/NeedList the Components of the SystemSpecify AssumptionsIdentify Relationships/InteractionsDraw a Diagram of the SystemCreate a Flowchart Diagram of the SystemConstruct the ModelVerify and Validate the ModelUse the ModelRevise the Model

The M&S ProcessFORMULATE PROBLEMDEVELOP COMPUTER MODEL DESIGN EXPERIMENTSIMULATION OUTPUT IN FORM OF OPERATING STATISTICSANALYZE SIMULATION RESULTSMAKE DESIRED CHANGES IN DECISION RULES, MODELPARAMETERS, OR SYS DESIGNVALIDATE MODELPERFORM SIMULATIONSIMULATION COMPLETE? SPECIFY PERFORMANCE CRITERIA, DECISION RULES, & CRITICAL SYSTEM PARAMETERSYESNO

The M&S ProcessFORMULATE PROBLEMDEVELOP MODEL DESIGN EXPERIMENTSIMULATION OUTPUT IN FORM OF OPERATING STATISTICSANALYZE SIMULATION RESULTSMAKE DESIRED CHANGES IN DECISION RULES, MODELPARAMETERS, OR SYS DESIGNVALIDATE MODELPERFORM SIMULATIONSIMULATION COMPLETE? SPECIFY PERFORMANCE CRITERIA, DECISION RULES, & CRITICAL SYSTEM PARAMETERSYESNO*

What are you modeling?Whats the System?Whats the Demarcation?What to Include?What to Exclude?Why are you modeling it?What are you looking for?Objectives? Goals?How will you Analyze it?How will you use the Output?FORMULATE PROBLEM SPECIFY PERFORMANCE CRITERIA, DECISION RULES, & CRITICAL SYSTEM PARAMETERS*Make the model only as complicated as it needs to be to address the issue of concern and to achieve the necessary level of fidelityThe M&S Process

Designing a ModelWhat to include? ... What to exclude? Size of the Dice? Color of Dice? Weight of Dice? Person rolling them?Surface they will roll on?Temperature of the room?

What are you modeling? Rolling Dice? A Dice Game?EXAMPLE*

Designing a ModelWhat to include? Glass? Current water level? Desired water level? Faucet position? Person filling glass?

What are you modeling? Filling a glass of water? Water flowing from a Faucet? EXAMPLEBased on: Senge, The Fifth DisciplineEXAMPLE

The M&S ProcessFORMULATE PROBLEMDEVELOP MODEL DESIGN EXPERIMENTSIMULATION OUTPUT IN FORM OF OPERATING STATISTICSANALYZE SIMULATION RESULTSMAKE DESIRED CHANGES IN DECISION RULES, MODELPARAMETERS, OR SYS DESIGNVALIDATE MODELPERFORM SIMULATIONSIMULATION COMPLETE? SPECIFY PERFORMANCE CRITERIA, DECISION RULES, & CRITICAL SYSTEM PARAMETERSYESNO*

Develop ModelEXAMPLEBased on: Senge, The Fifth DisciplineEXAMPLEDesired water level Current water levelFaucet positionGap betweenCurrent and DesiredLevel

Water FlowPersonDEVELOP MODELConceptual Model E.g., Causal Diagram More detailed Model as ApplicableNeeded for Simulation

The M&S ProcessFORMULATE PROBLEMDEVELOP MODEL DESIGN EXPERIMENTSIMULATION OUTPUT IN FORM OF OPERATING STATISTICSANALYZE SIMULATION RESULTSMAKE DESIRED CHANGES IN DECISION RULES, MODELPARAMETERS, OR SYS DESIGNVALIDATE MODELPERFORM SIMULATIONSIMULATION COMPLETE? SPECIFY PERFORMANCE CRITERIA, DECISION RULES, & CRITICAL SYSTEM PARAMETERSYESNO*Describe How you validated the ModelWhy?Test the models assumptionsTest model behavior and sensitivityDefine How Well you think the Model was VerifiedWhy?

The M&S ProcessFORMULATE PROBLEMDEVELOP MODEL DESIGN EXPERIMENTSIMULATION OUTPUT IN FORM OF OPERATING STATISTICSANALYZE SIMULATION RESULTSMAKE DESIRED CHANGES IN DECISION RULES, MODELPARAMETERS, OR SYS DESIGNVALIDATE MODELPERFORM SIMULATIONSIMULATION COMPLETE? SPECIFY PERFORMANCE CRITERIA, DECISION RULES, & CRITICAL SYSTEM PARAMETERSYESNO*Describe your Experiment(s)Inputs Generated, Expected OutputsWhy?Make sure this correlates with your Problem Statement

The M&S ProcessFORMULATE PROBLEMDEVELOP MODEL DESIGN EXPERIMENTSIMULATION OUTPUT IN FORM OF OPERATING STATISTICSANALYZE SIMULATION RESULTSMAKE DESIRED CHANGES IN DECISION RULES, MODELPARAMETERS, OR SYS DESIGNVALIDATE MODELPERFORM SIMULATIONSIMULATION COMPLETE? SPECIFY PERFORMANCE CRITERIA, DECISION RULES, & CRITICAL SYSTEM PARAMETERSYESNO*

The M&S ProcessFORMULATE PROBLEMDEVELOP MODEL DESIGN EXPERIMENTSIMULATION OUTPUT IN FORM OF OPERATING STATISTICSANALYZE SIMULATION RESULTSMAKE DESIRED CHANGES IN DECISION RULES, MODELPARAMETERS, OR SYS DESIGNVALIDATE MODELPERFORM SIMULATIONSIMULATION COMPLETE? SPECIFY PERFORMANCE CRITERIA, DECISION RULES, & CRITICAL SYSTEM PARAMETERSYESNO*Describe your Simulation ResultsShow Meaningful Plots

The M&S ProcessFORMULATE PROBLEMDEVELOP MODEL DESIGN EXPERIMENTSIMULATION OUTPUT IN FORM OF OPERATING STATISTICSANALYZE SIMULATION RESULTSMAKE DESIRED CHANGES IN DECISION RULES, MODELPARAMETERS, OR SYS DESIGNVALIDATE MODELPERFORM SIMULATIONSIMULATION COMPLETE? SPECIFY PERFORMANCE CRITERIA, DECISION RULES, & CRITICAL SYSTEM PARAMETERSYESNO*Test the models response to different situationsMake sure that your Experiments/Results correlate with your Problem Statement

The M&S ProcessFORMULATE PROBLEMDEVELOP MODEL DESIGN EXPERIMENTSIMULATION OUTPUT IN FORM OF OPERATING STATISTICSANALYZE SIMULATION RESULTSMAKE DESIRED CHANGES IN DECISION RULES, MODELPARAMETERS, OR SYS DESIGNVALIDATE MODELPERFORM SIMULATIONSIMULATION COMPLETE? SPECIFY PERFORMANCE CRITERIA, DECISION RULES, & CRITICAL SYSTEM PARAMETERSYESNO*Results !! Develop ConclusionsLessons Learned, Recommendations, Next Steps as appropriateModel Strengths/WeaknessesMake sure that your Experiments/Results correlate with your Problem Statement

Employs the cumulative distribution function (cdf) F(x) of the probability distribution under analysis to generate values of the random variable x.

CUMULATIVE DISTRIBUTION is defined over the interval (0, 1) and represents the probability that demand will be equal or less than x.

By RANDOMLY selecting values of r, we can randomly generate values of x according to the probability distribution of x.Process Generators for Modeling Processes

H .5T .5H .5H .5H .5H .5H .5H .5T .5T .5T .5T .5T .5T .5PROB # HEADS1/8 3 1/8 11/8 21/8 11/8 21/8 11/8 21/8 0 CONSTRUCT A PROBABILITY TREECumulative Distribution For The Outcomes Of Flipping A Coin 3 Times

Uniform random numbers can be generated by using mathematical functions called RANDOM NUMBERGENERATORS.TO GENERATE VARIATES FOR A DISCRETE RANDOM VARIABLE:(1) develop cumulative probability distribution (cdf) for the given random variable(2) use CDF to allocate the integer random numbers directly to the various values of the random variableRole of Random Numbers

Cumulative Distribution For The Outcomes of Flipping A CoinCONSTRUCT A PROBABILITY DISTRIBUTION TABLE# HEADS (X)P(X)CUMULATIVEPROBABILITYRANDOM # RANGE01231/83/83/81/81/8 = .1254/8 = .5007/8 = .8758/8 = 1.000 x

20%X = 14 20%X = 16 10%X = 17 10%X = 18 40%X = 15020809060The Monte Carlo Process

F(X)x14 15 16 17 18 1.0

.9 .8

.6

.2

0RANDOMNUMBER.90 1.0

.80 -

M&S is the most widely used problem-solving technique, excluding the disciplines of mathematics or statistics No alternative approaches for truly complex systemsSimulation/stimulation is the basis for testing software-intensive systemsSimulation-Based Acquisition is an accepted DoD practiceSimulation-Based Design is touted as a good practice in the system development communityOther reasons .Why is M&S Important to Systems Engineers?*

Verification versus ValidationVerification: 1. Determining that a simulation computer program performs as intended 2. Translating the flow chart and assumptions for a model into a correctly working program.Validation: Determining whether the conceptual simulation model (as opposed to the computer program) is an accurate representation of the system under study. (If a model is NOT valid, then any conclusions derived from the model will be of doubtful value.)

Validation, Verification, and Accreditation

Simulation results generated over a relatively short period of time are likely to be highly dependent on the sequence of random numbers generated, so they cannot be accepted as statistically valid. The simulation must be carried out over a long period of time to allow the results to converge on a correct solution.Number of Trials

Process Generators0 2 4 6 8 10 12 Frequencyb. Histogram0 0.5 1.0 0.25 0 0.25 0.50 0.75 1 a. Sorted Observations12 9 3 6 TimeBetweenTruckArrivals,Hours0 0.4 0.8 1.0 CumulativeProbabilityc. Cumulative Distribution Function0 0.5 1.0 0.25 0.75 0.75 0.2 0.6 Uniform Time Between Random Truck Arrivals Variable (hours) 0 < r < 0.40 0.25 0.40 < r < 0.70 0.50 0.70 < r < 0.80 0.75 0.80 < r < 1.00 1.00 d. Process GeneratorTime Between Truck Arrivals, Hours12/3021/3024/3030/30Time Between Truck Arrivals, Hours12212430

SERVER:Structure - Three ServersDistribution - Exponential747: = 4 AC/hr767: = 10 AC/hrMD88: = 20 AC/hrDiscipline: FCFS(Rerouted or Returned)QUEUE:Length - Unlimited (max of 10 will neverbe exceeded; Avg Arrivals = 10/hr)LOGPACS: 600 cubic feet747: 20 / AC767: 10 / ACMD88: 5 / AC STORAGE REQUIREMENTS?Simulation Example Processing CenterUNLOADINGEQUIPMENT?AIRCRAFT MIX?FLIGHT SCHEDULES?

Daves Candies is a small family-owned business that offers gourmet chocolates. For special occasions, you must place orders several weeks in advance. The St. Valentines Day Chocolate Massacre is bought for $7.50 and retails for $12. Any boxes not sold by Feb 14th are discounted 50%. Dave has sold between 40 and 90 boxes per year with no trend. Dave must order in increments of 10. How many shouldhe order?Daves Candies - Problem Statement

Process GeneratorIf there truly is not pattern then,

Demand Probability Role of Die 40 50 60 70 80 90 1/6 = .16671/6 = .16671/6 = .16671/6 = .16671/6 = .16671/6 = .1667123456Using this Information We Can Perform a Monte Carlo Simulation

Daves Profit Model

Determine order quantity Role the die (serves as a random number generator).Determine demand from model.Record the profit.Run enough times to assume a normal distribution.Assume OrderQuantityProfit ModelProfitRandomNumberN

Daves ResultsBased Upon the Results of 100 TrialsMean Values

Chart1

180

220.3846153846

247.0549450549

261.989010989

266.3736263736

256.7802197802

Order Size

Profit

Profit Versus Order Quantity

Excel Solution

Order Quantity

RandomSales405060708090

0.738699058877180225270315342327

0.840533231383180225270315360363

0.307418712156180225246231216201

0.432988630862180225270267252237

0.051729449243180183168153138123

0.997594566990180225270315360405

0.863966202684180225270315360369

0.178375280549180219204189174159

0.818885296981180225270315360351

0.2916653155180225240225210195

0.848162575583180225270315360363

0.606936533871180225270315306291

0.629233457972180225270315312297

0.195944406350180225210195180165

0.791816435580180225270315360345

0.658863269873180225270315318303

0.756614850778180225270315348333

0.585189786270180225270315300285

0.145558077348180213198183168153

0.868824775284180225270315360369

0.313097382456180225246231216201

0.306042125356180225246231216201

0.495282246665180225270285270255

0.32828518357180225252237222207

0.461637332864180225270279264249

0.491409335965180225270285270255

0.527087198467180225270297282267

0.978333564889180225270315360399

0.308461455356180225246231216201

0.22545782652180225222207192177

0.621435237872180225270315312297

0.69858407975180225270315330315

0.141459490448180213198183168153

0.583340062770180225270315300285

0.236616295452180225222207192177

0.539581725667180225270297282267

0.999415671890180225270315360405

0.254843268253180225228213198183

0.787308898380180225270315360345

0.671524679174180225270315324309

0.601843556571180225270315306291

0.409606060961180225270261246231

0.605353347671180225270315306291

0.898291192785180225270315360375

0.195869450350180225210195180165

0.980575106590180225270315360405

0.064727801544180189174159144129

0.904582733686180225270315360381

0.967600795889180225270315360399

0.257580226353180225228213198183

0.506634900466180225270291276261

0.37105066559180225264249234219

0.848758286483180225270315360363

0.975027179289180225270315360399

0.054513210943180183168153138123

0.977256536689180225270315360399

0.362829555559180225264249234219

0.32068629157180225252237222207

0.543680103568180225270303288273

0.885000835385180225270315360375

0.912761711686180225270315360381

0.663844629674180225270315324309

0.796274097980180225270315360345

0.453586813463180225270273258243

0.690328443875180225270315330315

0.906351695686180225270315360381

0.575339424169180225270309294279

0.983465899690180225270315360405

0.07331929544180189174159144129

0.073309041444180189174159144129

0.656330700873180225270315318303

0.624751735572180225270315312297

0.115631663146180201186171156141

0.720164410577180225270315342327

0.266137347554180225234219204189

0.21944233851180225216201186171

0.01355081941180171156141126111

0.29460599855180225240225210195

0.194555967750180225210195180165

0.776252865579180225270315354339

0.549707944768180225270303288273

0.359138501158180225258243228213

0.888696864585180225270315360375

0.675457130274180225270315324309

0.470713840664180225270279264249

0.546259486168180225270303288273

0.655388017873180225270315318303

0.909760114186180225270315360381

0.051257180643180183168153138123

0.686056451775180225270315330315

0.894204210885180225270315360375

Quantity405060708090

Average$180.00$221.24$251.67$272.08$278.57$272.67

Daves Candies is a small family-owned business that offers gourmet chocolates. For special occasions, you must place orders several weeks in advance. The St. Valentines Day Chocolate Massacre is bought for $7.50 and retails for $12. Any boxes not sold by Feb 14th are discounted 50%. Dave has sold between 40 and 90 boxes per year with no trend. Dave must order in increments of 10. How many should he order?

Excel Solution

180

220.3846153846

247.0549450549

261.989010989

266.3736263736

256.7802197802

Order Size

Profit

Profit Versus Order Quantity

CB Solution

Sheet3

Class Problem 1As a newly assigned Operations Engineer, you have developed a Self Service Supply Center (SSSC) for your factory. You have one supply clerk to process the paper work for the customers. Classic InfiniteQueuing ProblemEvent Driven Simulation

Manual Simulation of Class SSSC

Custr1IA TimeTotalAtWaitQueuer2ServiceDepartTime#ClockCashierTimeLengthTimeClockInTimeTimeAfter EntrySystem

100000.0611120.93333000.4725230.72255000.316140.83388000.97311350.35199210.96314560.1711010420.2115570.8231313220.16116380.6321515110.5218390.6821717110.031192100.9842121010.071221

10.82.6

Summary - BenefitsSolve a wide variety of complex problemsIncorporate risk analysisCapture variability of many inputs, outputs, and processesLimited optimization abilitiesMost widely used decision tool

Summary Limitations Most abused decision science tool Cannot be used for optimization Data is more difficult to develop than model Can be hard to explain risk and stochastic variability to decision makers

Questions?

********5443333333334333*6443333333334333*7443333333334333*5443333333334333*************8443333333334333*