1

Introduction to SimulationChapters 24

OverviewSimulation: Key QuestionsIntroduction to Simulation Common Mistakes in Simulation Other Causes of Simulation Analysis FailureChecklist for SimulationsTerminologyTypes of Models

Simulation: Key QuestionsWhat are the common mistakes in

simulation and why most simulations fail? What language should be used for

developing a simulation model?What are different types of simulations?How to schedule events in a simulation? How to verify and validate a model? How to determine that the simulation has

reached a steady state?How long to run a simulation?

Simulation: Key Questions (Cont)How to generate uniform random numbers?How to verify that a given random number

generator is good?How to select seeds for random number

generators? How to generate random variables with a

given distribution?What distributions should be used and

when?

Introduction

5

System to be characterized may not be availableDuring design or procurement stage

Still want to predict performanceOr, may have system but want to evaluate

wide-range of workloads Simulation

The best advice to those about to embark on a very large simulation is often the same as Punch’s famous advice to those about to marry: ‘Don’t!’

– Bratley, Fox and Schrage (1986)

Simulation Failuresimulation models often fail; that is, they

produce no useful results or misleading results.

This is either because the model developers are proficient in software development but lack statistical background or because they are proficient in statistical techniques but are not aware of good software development techniques

Common Mistakes in Simulation (1 of 4)

7

1. Inappropriate level of detailLevel of detail often potentially unlimited But more detail requires more time to develop

And often to run!Can introduce more bugs, making more inaccurate

not less!Often, more detailed viewed as “better” but may

not be the caseMore detail requires more knowledge of input

parametersGetting input parameters wrong may lead to more

inaccuracy (Ex: disk service times exponential vs. simulating sector and arm movement)

Start with less detail, study sensitivities and introduce detail in high impact areas

Common Mistakes in Simulation (2 of 4)

8

2. Improper languageChoice of language can have significant impact

on time to developSpecial-purpose languages can make

implementation, verification and analysis easierC++Sim, JavaSim

3. Unverified modelsSimulations generally large computer programsUnless special steps taken, bugs or errors

Common Mistakes in Simulation (3 of 4)

9

4. Invalid modelsNo errors, but does not represent real systemNeed to validate models by analytic,

measurement or intuition

5. Improperly handled initial conditionsOften, initial trajectory not representative of

steady stateIncluding can lead to inaccurate results

Typically want to discard, but need method to do so effectively

Common Mistakes in Simulation (4 of 4)

10

6. Too short simulation runsAttempt to save timeMakes even more dependent upon initial

conditionsCorrect length depends upon the accuracy

desired (confidence intervals)7. Poor random number generators

“Home grown” are often not random enoughBest to use well-known one

8. Improper Selection of Seeds:

More Causes of Failure

11

Large softwareQuotations above apply to software development

projects, including simulationsIf large simulation efforts not managed properly,

can fail1. Inadequate time estimate

Need time for validation and verificationTime needed can often grow as more details

added

Any given program, when running, is obsolete. If a program is useful, it will have to be changed. Program complexity grows until it exceeds the capacity of the programmer who must maintain it. - Datamation 1968

More Causes of Failure

12

2. No achievable goalCommon example is “model X”

But there are many levels of detail for XGoals: Specific, Measurable, Achievable,

Repeatable, Through (SMART)The goals should be clearly written down and

agreed to by the analysts and the end users of the results before beginning the model development

Project without goals continues indefinitely

More Causes of Failure3. Incomplete mix of essential skills

A simulation project requires at least the following four areas of skills:

(a)  Project Leadership: The ability to motivate, lead, and manage the members of the simulation team.

(b)  Modeling and Statistics: The ability to identify the key characteristics of the system and model them at the required level of detail.

(c)  Programming: The ability to write a readable and verifiable computer program that implements the model correctly.

(d)  Knowledge of the Modeled System: The ability to understand the system, explain it to the modeling team, and interpret the modeling results in terms of their impact on the system

Other Causes of Simulation Analysis Failure

4. Inadequate Level of User Participation: You need to have meeting with the user to make sure the system is not changing.

5. Obsolete or Nonexistent Documentation: Do the documentation during the simulation

6. Inability to Manage the Development of a Large Complex Computer: Program Need software engineering tools to manage it

7. Mysterious Results: Most mysterious results are due to bugs in the simulation program, invalid model assumptions, or lack of understanding of the real system.

Simulation Checklist (1 of 2)

15

Simulation Checklist (2 of 2)

16

Terminology (1 of 7)

17

Introduce terms using an example of simulating CPU schedulingStudy various scheduling techniques given job

characteristics, ignoring disks, display…State variables

Variables whose values define current state of system

Saving can allow simulation to be stopped and restarted later by restoring all state variables

Ex: In the CPU scheduling simulation, the state variable is the length of the job queue.

Ex: in M/M/1 simulation, we will write down the queue length

Terminology (2 of 7)

18

EventA change in system state is called eventEx: In the CPU scheduling simulation, Three

events: arrival of job, beginning of new execution, departure of job

Continuous-time and discrete-time modelsIf state defined at all times continuousIf state defined only at instants discreteEx: class that meets M-F 2-3 is discrete since

not defined other times

Job

s

Time Stu

den

ts

Time

Terminology (3 of 7)

19

Continuous-state and discrete-state modelsIf uncountably infinite continuous

Ex: time spent by students on hwIf countable discrete

Ex: jobs in CPU queueNote, continuous time does not necessarily

imply continuous state and vice-versaAll combinations possible

Job

s

Time

Tim

e

Term

Types of Models (Cont)

Terminology (4 of 7)

21

Deterministic and probabilistic modelsIf output predicted with certainty

deterministicIf output different for different repetitions

probabilisticEx: Simulation is usually probabilistic if you

change the seed. It is better to repeat it.Otherwise it is deterministic

Ou

tpu

t

Input

Ou

tpu

t

Input

(Deterministic) (Probabilistic)

(vertical lines)

Terminology (5 of 7)

22

Static and dynamic modelsIf Time is not a variable staticIf state changes with time dynamicEx: CPU scheduler is dynamic, while matter-to-

energy model E=mc2 is staticLinear and nonlinear models

If Output is linear combination of input linearOtherwise nonlinear

Ou

tpu

t

Input(Linear)

Ou

tpu

t

Input(Non-Linear)

Terminology (6 of 7)

23

Open and closed modelsInput is external and independent openClosed model has no external inputEx: if same jobs leave and re-enter queue

then closed, while if new jobs enter system then open

cpu

open

cpu

closed

Terminology (7 of 7)

24

Stable and unstableModel output settles down and reaches steady

state stableModel output always changes unstableHow about Sinusoidal Model?

Ou

tpu

t

Time

(Unstable)

Ou

tpu

t

Time

(Stable)

Computer System ModelsContinuous time Discrete stateProbabilisticDynamic NonlinearOpen or closed Stable or unstable

HW

Selecting a Language for Simulation

Simulation Languages

General Purpose LanguageAnalyst's familiarity Easy availabilityQuick startup: You know the programming

languageTime for routines for event handling,

random number generation Other Issues: Efficiency, flexibility, and

portability Recommendation: Learn at least one

simulation language.

Extensions of a General Purpose Language

Simulation PackagesExamples: Opnet, NS2Input dialog Library of data structures, routines, and

algorithmsBig time savingsProblem: Inflexible

Types of Simulation Languages

Discrete Event Simulation Example NS - (1 of 4)

33

NS-2, network simulatorGovernment funded initially, Open source

Wildly popular for IP network simulations

Discrete Event Simulation Example NS - (2 of 4)

34

(Event scheduler is core of simulator)

Discrete Event Simulation Example NS - (3 of 4)

35

# Open the NAM trace fileset nf [open out.nam w]$ns namtrace-all $nf

# Define a 'finish' procedureproc finish {} { global ns nf $ns flush-trace # Close the trace file close $nf #Execute NAM on file exec nam out.nam & exit 0}

# Setup a FTPset ftp [new Application/FTP]$ftp attach-agent $tcp$ftp set type_ FTP

# Initial schedule events $ns at 0.1 "$cbr start"$ns at 1.0 "$ftp start"$ns at 4.0 "$ftp stop"$ns at 4.5 "$cbr stop"

# Finish after 5 sec (sim time)$ns at 5.0 "finish"

# Run the simulation$ns run

Discrete Event Simulation Example NS - (4 of 4)

36

Output in text file, can be processed with Unix command line tools

(Objects and script can have custom output, too)