1
Introduction to SimulationChapters 24
OverviewSimulation: Key QuestionsIntroduction to Simulation Common Mistakes in Simulation Other Causes of Simulation Analysis FailureChecklist for SimulationsTerminologyTypes of Models
Simulation: Key QuestionsWhat are the common mistakes in
simulation and why most simulations fail? What language should be used for
developing a simulation model?What are different types of simulations?How to schedule events in a simulation? How to verify and validate a model? How to determine that the simulation has
reached a steady state?How long to run a simulation?
Simulation: Key Questions (Cont)How to generate uniform random numbers?How to verify that a given random number
generator is good?How to select seeds for random number
generators? How to generate random variables with a
given distribution?What distributions should be used and
when?
Introduction
5
System to be characterized may not be availableDuring design or procurement stage
Still want to predict performanceOr, may have system but want to evaluate
wide-range of workloads Simulation
The best advice to those about to embark on a very large simulation is often the same as Punch’s famous advice to those about to marry: ‘Don’t!’
– Bratley, Fox and Schrage (1986)
Simulation Failuresimulation models often fail; that is, they
produce no useful results or misleading results.
This is either because the model developers are proficient in software development but lack statistical background or because they are proficient in statistical techniques but are not aware of good software development techniques
Common Mistakes in Simulation (1 of 4)
7
1. Inappropriate level of detailLevel of detail often potentially unlimited But more detail requires more time to develop
And often to run!Can introduce more bugs, making more inaccurate
not less!Often, more detailed viewed as “better” but may
not be the caseMore detail requires more knowledge of input
parametersGetting input parameters wrong may lead to more
inaccuracy (Ex: disk service times exponential vs. simulating sector and arm movement)
Start with less detail, study sensitivities and introduce detail in high impact areas
Common Mistakes in Simulation (2 of 4)
8
2. Improper languageChoice of language can have significant impact
on time to developSpecial-purpose languages can make
implementation, verification and analysis easierC++Sim, JavaSim
3. Unverified modelsSimulations generally large computer programsUnless special steps taken, bugs or errors
Common Mistakes in Simulation (3 of 4)
9
4. Invalid modelsNo errors, but does not represent real systemNeed to validate models by analytic,
measurement or intuition
5. Improperly handled initial conditionsOften, initial trajectory not representative of
steady stateIncluding can lead to inaccurate results
Typically want to discard, but need method to do so effectively
Common Mistakes in Simulation (4 of 4)
10
6. Too short simulation runsAttempt to save timeMakes even more dependent upon initial
conditionsCorrect length depends upon the accuracy
desired (confidence intervals)7. Poor random number generators
“Home grown” are often not random enoughBest to use well-known one
8. Improper Selection of Seeds:
More Causes of Failure
11
Large softwareQuotations above apply to software development
projects, including simulationsIf large simulation efforts not managed properly,
can fail1. Inadequate time estimate
Need time for validation and verificationTime needed can often grow as more details
added
Any given program, when running, is obsolete. If a program is useful, it will have to be changed. Program complexity grows until it exceeds the capacity of the programmer who must maintain it. - Datamation 1968
More Causes of Failure
12
2. No achievable goalCommon example is “model X”
But there are many levels of detail for XGoals: Specific, Measurable, Achievable,
Repeatable, Through (SMART)The goals should be clearly written down and
agreed to by the analysts and the end users of the results before beginning the model development
Project without goals continues indefinitely
More Causes of Failure3. Incomplete mix of essential skills
A simulation project requires at least the following four areas of skills:
(a) Project Leadership: The ability to motivate, lead, and manage the members of the simulation team.
(b) Modeling and Statistics: The ability to identify the key characteristics of the system and model them at the required level of detail.
(c) Programming: The ability to write a readable and verifiable computer program that implements the model correctly.
(d) Knowledge of the Modeled System: The ability to understand the system, explain it to the modeling team, and interpret the modeling results in terms of their impact on the system
Other Causes of Simulation Analysis Failure
4. Inadequate Level of User Participation: You need to have meeting with the user to make sure the system is not changing.
5. Obsolete or Nonexistent Documentation: Do the documentation during the simulation
6. Inability to Manage the Development of a Large Complex Computer: Program Need software engineering tools to manage it
7. Mysterious Results: Most mysterious results are due to bugs in the simulation program, invalid model assumptions, or lack of understanding of the real system.
Simulation Checklist (1 of 2)
15
Simulation Checklist (2 of 2)
16
Terminology (1 of 7)
17
Introduce terms using an example of simulating CPU schedulingStudy various scheduling techniques given job
characteristics, ignoring disks, display…State variables
Variables whose values define current state of system
Saving can allow simulation to be stopped and restarted later by restoring all state variables
Ex: In the CPU scheduling simulation, the state variable is the length of the job queue.
Ex: in M/M/1 simulation, we will write down the queue length
Terminology (2 of 7)
18
EventA change in system state is called eventEx: In the CPU scheduling simulation, Three
events: arrival of job, beginning of new execution, departure of job
Continuous-time and discrete-time modelsIf state defined at all times continuousIf state defined only at instants discreteEx: class that meets M-F 2-3 is discrete since
not defined other times
Job
s
Time Stu
den
ts
Time
Terminology (3 of 7)
19
Continuous-state and discrete-state modelsIf uncountably infinite continuous
Ex: time spent by students on hwIf countable discrete
Ex: jobs in CPU queueNote, continuous time does not necessarily
imply continuous state and vice-versaAll combinations possible
Job
s
Time
Tim
e
Term
Types of Models (Cont)
Terminology (4 of 7)
21
Deterministic and probabilistic modelsIf output predicted with certainty
deterministicIf output different for different repetitions
probabilisticEx: Simulation is usually probabilistic if you
change the seed. It is better to repeat it.Otherwise it is deterministic
Ou
tpu
t
Input
Ou
tpu
t
Input
(Deterministic) (Probabilistic)
(vertical lines)
Terminology (5 of 7)
22
Static and dynamic modelsIf Time is not a variable staticIf state changes with time dynamicEx: CPU scheduler is dynamic, while matter-to-
energy model E=mc2 is staticLinear and nonlinear models
If Output is linear combination of input linearOtherwise nonlinear
Ou
tpu
t
Input(Linear)
Ou
tpu
t
Input(Non-Linear)
Terminology (6 of 7)
23
Open and closed modelsInput is external and independent openClosed model has no external inputEx: if same jobs leave and re-enter queue
then closed, while if new jobs enter system then open
cpu
open
cpu
closed
Terminology (7 of 7)
24
Stable and unstableModel output settles down and reaches steady
state stableModel output always changes unstableHow about Sinusoidal Model?
Ou
tpu
t
Time
(Unstable)
Ou
tpu
t
Time
(Stable)
Computer System ModelsContinuous time Discrete stateProbabilisticDynamic NonlinearOpen or closed Stable or unstable
HW
Selecting a Language for Simulation
Simulation Languages
General Purpose LanguageAnalyst's familiarity Easy availabilityQuick startup: You know the programming
languageTime for routines for event handling,
random number generation Other Issues: Efficiency, flexibility, and
portability Recommendation: Learn at least one
simulation language.
Extensions of a General Purpose Language
Simulation PackagesExamples: Opnet, NS2Input dialog Library of data structures, routines, and
algorithmsBig time savingsProblem: Inflexible
Types of Simulation Languages
Discrete Event Simulation Example NS - (1 of 4)
33
NS-2, network simulatorGovernment funded initially, Open source
Wildly popular for IP network simulations
Discrete Event Simulation Example NS - (2 of 4)
34
(Event scheduler is core of simulator)
Discrete Event Simulation Example NS - (3 of 4)
35
# Open the NAM trace fileset nf [open out.nam w]$ns namtrace-all $nf
# Define a 'finish' procedureproc finish {} { global ns nf $ns flush-trace # Close the trace file close $nf #Execute NAM on file exec nam out.nam & exit 0}
# Setup a FTPset ftp [new Application/FTP]$ftp attach-agent $tcp$ftp set type_ FTP
# Initial schedule events $ns at 0.1 "$cbr start"$ns at 1.0 "$ftp start"$ns at 4.0 "$ftp stop"$ns at 4.5 "$cbr stop"
# Finish after 5 sec (sim time)$ns at 5.0 "finish"
# Run the simulation$ns run
Discrete Event Simulation Example NS - (4 of 4)
36
Output in text file, can be processed with Unix command line tools
(Objects and script can have custom output, too)