1

Chapter 10Sample path sensitivity analysis

Learning objectives :• Understand the mathematics behind discrete event

simulation• Learn how to derive gradient information during a simulation• Able to verify the correctness of sample gradients

Textbook :C. Cassandras and S. Lafortune, Introduction to Discrete Event

Systems, Springer, 2007

2

Plan

• Introduction• Perturbation analysis by example• GSMP - Generalised Semi-Markovian Process• Some mathematical basis of discrete event simulation• Perturbation analysis of GSMP• Correctness of gradient estimators• Applications to manufacturing systems• Sensitivity estimation revisited• Extensions of perturbation analysis

3

Introduction

4

Sensitivity measures

Two types of sensitivity measures:•Derivatives : d J()/d•Finite difference : J() - J()where J() is the performance funtion of a system at parameter

Sensitivity measures are for•determination of the direction of change to take•optimization of a system design•performing "what-if" analysis

5

Sensitivity measures

Difficulties of sensitivity analysis:•Unknown performance function J()•Estimation by simulation in most cases

Main ideas of perturbation analysis•Prediction of the behavior of perturbed system along the simulation of the system

Main steps of perturbation analysis•Perturbation generation: changes in quantities directly linked to •Perturbation propagation: changes in other quantities generated by perturbation generation

6

Stochastic approximation

Stochastic approximation (gradient based minimization) :x(n+1) = x(n) - a(n)g(n)

where •x(n) : value of x at iteration n•a(n) > 0 : stepsize•g(n) : gradient estimation of a performance function f(x)

Round-Robin Theorem: Assume that f(x) is continuously differentiable and there exists a single x* such that df(x*)/dx = 0. If

E[g(n)] = df(x)/dxand

limn a(n) = 0, n a(n) = ∞then x(n) converges with probability 1 to x*.

7

Perturbation analysis by example

8

An inspection machine

• Products arrive randomly to an inspection machine• Each product requires a random number of tests• The inspection of a production requires a setup of time • Each test takes time units• Products are inspected in FIFO order• The buffers are of unlimited capacity• The system is empty initially

• Performance measure = T(), average time at the inspection station.

Input Buffer

Inspection Station

Ouput Buffer

Arrival

9

A G/G/1 queueing model

Service time X = + L

Inter-arrival time A

10

Sample path

Definition : A Sample Path is the sequence (event, state, time) of a simulation or experimentation.

Example :(Start, 0, 0), (arrival, 1, A1), (arrival, 2, A1+A2),(arrival, 3, A1+A2+A3), (departure, 2, A1+X1), …

Property : The sample path contains all information of the simulation.

A1

P1

A2 A3

P2 P3 P4 P5 P6

A4 A5

X1 X2 X3 X4

P1 P2 P3 P4 Time

0

Buffer level

11

An inspection machine

Each simulation run or a sample path is characterized by :•Li : number of tests of product i

•Xi = + Li : service time of product i

•Ai : inter-arrival time between product i and product i-1

•ti : time at the inspection station of product i (also called system time)

Attention ti ≠ Xi

Input Buffer

Inspection Station

Ouput Buffer

Arrival

12

An inspection machine

The best estimator of the average time at the inspection station:

Under some regularity conditions

Goal of perturbation analysisEvaluate ∂T()/∂ and ∂T()/∂

Input Buffer

Inspection Station

Ouput Buffer

Arrival

ˆ T ,

1

N t i

i 1

N

limN

ˆ T , T , , w.p.1

13

Event-based system dynamics

{e1, e2, ..., en, ...} : sequence of events with en {a, d}

n : occurrence time of en with n = 0

qn : number of parts in the system at time n+ with qn = 0

n : time in state qn

En : set of active events at n+

ren : remaining time till occurrence of event e at time n+

14

Event-based system dynamics

en : sequence of eventsn : occurrence time of en

qn : number of parts at time n+n : time in state qn

En : set of active events at n+ren : remaining time of event e

Time to next event :n = Min (ren , eEn)Next event : en+1 = Argmin (ren , eEn)New state

qn+1 = qn +1 if en+1 = aqn+1 = qn -1 if en+1 = d

Update event clocksren+1 = ren - n

En+1 = En -{en+1}Create new eventCase e = a and less than N arrivalsEn+1 = En+1 +{a} ran+1 = An+1

Case e = d and qn+1 > 0 ANDCase e = a and qn = 0 En+1 = En+1 +{d} ran+1 = +Ln+1

15

Event-based system dynamics

en : sequence of eventsn : occurrence time of en

qn : number of parts at time n+n : time in state qn

En : set of active events at n+ren : remaining time of event e

Performance evaluation :

Sn = total time of all parts in the system up to event n

Sn+1 = Sn + qnn

T() = SV /N

where eV is the departure event of N-th part

16

Event-based system dynamics

Simulation algorithm

1/ Initialization : q = 0, t = 0, E = {a}, generate r.v. A, ra = ANa = Nd = S = 0

2/ Next event : e = Argmin (re, eE), = re

Statistics : Na = Na+ 1(e=a), Nd = Nd+ 1(e=d), S = S + q

3/ State update : q = q + 1(e = a)-1(e= d)

4/ Update event clocks : re = re - , E = E -{e}

5/ Generate new eventsIf e=a Na < N, E = E +{a}, generate r.v. A, ra = AIf (e=a q= 1) or (e=d Nd < N), E = E +{d}, generate r.v. L, rd = +L

17

Perturbation analysis

Typical question (What-If) :

What would be the average system time

if the experimentation were performed with parameter ? Definitions :

•Nominal system : the system with parameter of the simulation•Perturbed system : the system with parameter of the What-If question?•Nominal sample path : sample path of the nominal system•Perturbed sample path : sample path of the perturbed system (preferably under the same random condition as the nominal sample path 

18

Perturbation analysis

Typical perturbation analysis answer:

•Perturbation generationThe perturbation of parameter generates directly a perturbation ∆Xi

= Li∆ to each product service time (recall that Xi = + Li)

•Perturbation propagationThe perturbation of the service time of a product will be propagated

to other products

•Fundamental PA condition : the sequence of events does not change for small enough perturbation

19

Perturbation analysis

A1

P1

A2 A3

P2 P3 P4 P5 P6

A4 A5

X1 X2 X3 X4

P1 P2 P3 P4

Nb of products

Time 0

X1 X2 X3 X4

20

Event-based perturbation analysis

Next event : en+1 =e*= Argmin (ren , eEn)

Time to next event : n = re*n , n / = re*n /

New state qn+1

Update event clocks ren+1 = ren - n , ren+1/ = ren+1 / - n/ En+1 = En -{en+1}

Create new eventNew arrival : En+1 = En+1 +{a}, ran+1 = An+1, ran+1/ = 0New depart : En+1 = En+1 +{d}, rdn+1 = + Ln+1, rdn+1/ = Ln+1

Statistics : Sn+1 = Sn + qnn, Sn+1 / = Sn / + qnn/

21

Event-based system dynamics

Simulation algorithm1/ Initialization : q = 0, t = 0, E = {a}, generate r.v. A, ra = ANa = Nd = S = 0, grad_ra = 0

2/ Next event : e = Argmin (re, eE), = re , grad_ = grad_re

Statistics : Na = Na+ 1(e=a), Nd = Nd+ 1(e=d), S = S + qgrad_S = grad_S + q*grad_

3/ State update : q = q + 1(e = a)-1(e= d)

4/ Update event clocks : E = E -{e}, re = re - , grad_re = grad_re - grad_

5/ Generate new eventsIf e=a Na < N, E = E +{a}, generate A, ra = A, grad_ra = 0If (e=a q= 1) or (e=d Nd < N), E = E +{d}, generate L, rd = +Lgrad_rd = L

6/ If Nd < N, go to 2; otherwise, T = S/N, grad_T = grad_S /N, END

22

Busy period

Definition : A busy period (BP) is a period of time during which the inspection station is never idle.

For a product i of the 1st busy period (BP1):• arrival date : A1+A2 + ...+Ai

• departure time: A1+ X1+X2 + ...+Xi

• system time : ti = Xi + (X1+X2 + ...+Xi-1) - (A2 + ...+Ai)

Total system time of products of BP1

X jj1

i

A jj 2

i

i1

4

23

Busy period

• Each busy period BPm is initiated by the arrival of a product that finds an empty system

• Let km +1 be the product that initiates busy period BPm• System time of the i-th product of BPm

• Total system time of products of BPm

where nm is the number of products inspected in BPm

1 2m m

i i

i k j k jj j

t X A

1 1 2

m

m m

n i i

k j k ji j j

X A

24

Busy period

• Total system time of products of BPm

where M is the total number of busy periods observed during the simulation

• M is a random variable that depends on the simulation realization.

1 1 1 2

1,

m

m m

nM i i

k j k jm i j j

T X AN

25

Busy period-based Perturbation analysis

• Busy periods remain the same dnder the fundamental condition.

• System time in the perturbed system

• As a result,

1 2 1

m m m

i i i

i k j k j i k jj j j

t X A t X

1 1

m

m

i ik ji

k jj j

XtL

1 1 1 1

ˆ 1 1 m

m

nN M ii

k ji m i j

tTL

N N

26

Busy period-based Perturbation analysis

• To summarize, for Xi = + Li,

1 1

ˆ 1, with

m

N ii i

ji j k

t tTL

N

1 1

ˆ 1, with 1

m

N ii i

i j k

t tT

N

27

Busy period-based Perturbation analysis

Perturbation analysis algorithm during the simulation

0) Initialization : gradX = 0, gradt = 0, gradT = 0 gradX = 0, gradt = 0, gradT = 0

1) During the simulation : If the end of service of product with L tests,1.1) Perturbation generation :

gradX L and gradX11.2) Perturbation propagation

(system time) gradt = gradt+ gradX(Total system time) gradT = gradTgradt

gradt = gradt+ gradXgradT = gradTgradt

1.3) If the system is empty, thengradt = 0 and gradt = 0

2) At the end of the simulation,∂T/∂ = gradT/N and ∂T/∂ = gradT/N

28

Validation of the results

Unbiasedness

Consistency

E T , , N

?

ET , , N

E T , , N

?

ET , , N

N lim T , , N

?

T , ,

N lim T , , N

?

T , ,

29

GSMP - Generalised Semi-Markovian Process

A general framework for representation of Discrete event systems

30

Basic concepts

States : a state is a possible system configuration Ex : number of products to inspect

Events : the system state changes only upon the occurrence of eventsEx : arrival and end of inspection of a product

Transition probabilities : which rule the change of the state when an event occurs Ex : routing probabilities among multiple alternatives

Clock : which indicate the remaining time to occurrence of events. • A clock is set according to some random variable when it is activated • It then decrements as long as it is active. • An event occurs when its clock reaches zero. Attention : Remaining processing times of products are associated with

clocks of related events and not in the definition of the state

31

GSMP

A GSMP is defined by :

• S : set of states

• E : set of events

• E(s) : set of events that are possible at state s

• p(s'; s, e) : probability of jumping to state s' when event e occurs at state s

• Fe : probability distribution of a new clock of event e

Example (Station d'inspection) :

• S = {0, 1, …} : number of products waiting for inspection

• E = {a, d} with a = product arrival, d = product departure

• E(s) = {a, d} if s > 0 and E(0) = {a}

• p(s+1; s, a) = 1, p(s-1; s, d) d 1

• Fa = distribution of inter-arrival time, Fd = distribution of service time

32

GSMP

Non-Interruption (NI) condition :

s, s' �S and e � E(s), p(s'; s, e) > 0 E(s')   E(s) - {e}

The NI condition implies that, once activated, an event remain active till its occurrence.

33

Simulation algorithm of an GSMP

1. Initialization :• Number of events : n = 0• Set initial state : S0• Set clocks of activated events : C0(e) = X(e, 1), e E(S� 0)• Set event counter : N(e, 0) = 0, e E�

2. Determine the next event :

3. Sojourn time at Sn : n+1 = Cn(en+1)

4. Update the event counters :

e n1 arg mine E s

C n e

1, 1, if, 1

, ,otherwise

nN e n e eN e n

N e n

34

Simulation algorithm of an GSMP

5. Update the system state :Sn+1 = (Sn , en+1 , U(en+1, N(en+1, n+1)))where • is a function such that P{(s, e, U) = s'} = p(s'; s, e) • U is a random variable with uniform distribution on [0,1].

6. Update existing clocks :Cn+1(e) = Cn+1(e) - n+1, e E(sn) -{en+1}

7. Generate new clocks :Cn+1(e) = X(e, N(e, n) +1), e E(sn+1) - (E(sn) -{en+1})

8. If the end of simulation, stop. Otherwise, n:= n+1 and go to step 2

35

Simulation algorithm of an GSMPRouting mechanisms

• Each active event e is associated a uniform random variable U(e), called routing indicator

Step 5 of the GSMP simulation algorithm is decompose into two steps:

• 5.1. Generation of the routing indicator : U(en+1,N(en+1, n+1))

• 5.2. Determination of the new state :

Sn+1 = (Sn , en+1 , U(en+1,N(en+1, n+1)) )

where is such that P{(s, e, U) = s'} = p(s'; s, e). Example of :Let S = { s1, s2, … , sm }. Sn+1 := si such that : p s j ; Sn , e n1

j1

i 1

U p s j ; Sn , e n1 j1

i

36

Simulation algorithm of an GSMPPerformance measures

LetZt : be the state of the system at time t andf(Zt) : the cost associated with state Zt .  The following performance measures are considered

•

where T is a given constant

•

where N is a given integer and tN is occurrence time of N-th event, i.e. tN = ;

•

where T(,k) is the time of k-th occurrence of event a .

L T f Z t dt0T

L N f Z t dt0tN

L,k f Z t dt0T ,k

37

Some mathematical basis of discrete event simulation

38

Random Number GenerationThe linear congruential technique

Goal:• Generate samples of U[0, 1] distribution

The linear congruential technique :Xk+1 = (aXk + c) mod m, k = 0, 1, ...

wherea = multiplier, c = increment, m = modulusX0 = seed

Quality strongly depends on the selection of parameters a, c, m.

The "pseudo" random samples have cycle of length at most m.

m = 2n , as large as possible with 231 for 32-bit computer, is often used to maximize the cycle and to ease to the modula arithmetic

39

Random variate generationThe inverse transform technique

Goal:• Generate samples of probability distribution F(x)

The inverse tranform technique :• Generate a sample U of distribution U[0, 1]• Transform U into X with X = F-1(U)

Online proof

Examples : • EXP() with F = 1 - exp(-x)• Weibull with F = 1 - exp (-((x-)))

Drawback: • The computation of the inverse transform can be time consuming.

40

Random variate generationAcceptance-rejection technique

Majorizing function:• g(x) such that g(x) ≥ f(x)Density function related to g(x)

Acceptance-rejection technique :1. Select a majorizing function g(x) and its

related density function h(x)2. Generate a random variate Y with pdf h(x)3. Generate a random number U[0,1]

independently from Y4. If U ≤ f(Y)/g(Y),

then set X = Y. Otherwise, repeat step 2.

h x g x g y dy

Criteria for selection of majorizing function:•Ease of generation of distribution h(x)•Small rejection points, i.e. small surface between g(x) and f(x)

41

Terminating simulation output analysisPoint estimation

Observations:• X1, X2, ..., Xn : simulation observations of true but unknown performance

Sample mean

Unbiasedness:

For iid observations X1, X2, ..., Xn,

Sample variance:

1

1ˆn

n ii

Xn

n̂E

2

ˆ ˆ ˆ, , limn i n nnE E X Var

n

22

1

1 ˆ1

n

n i ni

S Xn

strong law of large number

42

Terminating simulation output analysisInterval estimation

Central limit theorem

tends to a standard normal distribution as n goes to infinity.

(1-)-percent confidence interval (why):

ˆ ˆ

ˆ

n n

n

n

EZ

Var

2 2/ 2 / 2

ˆ ˆ/ / 1n nP z n z n

where• z/2 is a defined on a standard normal

distribution F(z).• 2 is evaluated by sample variance

Student's t distribution is also used to avoid checking how large n is for normal distribution approximation

43

Nonterminating simulation output analysisProblem setting

Observations:• X1, X2, ... : simulation observations of a nonterminating simulation

Unknown steady state performance :

Time average:

Under ergodicity condition

lim kkE X

1

1lim

n

ini

Xn

1

1lim

n

ini

Xn

44

Nonterminating simulation output analysisReplications with deletions

I.• Simulation for a time horizon of m+r steps• Ignore data of the first r steps• Estimation with data of the remaining m steps

II.• Perform K replications of I• Use point estimation methods to evaluate the performance

Key questions :• How long should the warmup interval r be?

,1

1ˆm

m r ii r

Xm r

45

Nonterminating simulation output analysisBatch means

• Based on a single but long simulation run

• Observed data X1, X2, ... are grouped into Batches

Assume batches of equal length m with a warmup period of length r

• Batch mean Bj is estimated

• Batch means are then used in point estimation

( 1) 1

1 r jm

j ii r j m

B Xm

1

1ˆn

n jj

Bn

46

Nonterminating simulation output analysisRegenerative simulation

• Assumption:

There exists some regenerative state (regeneration point) after which the system is independent of past history

• Regeneration cycles

R1, R2, ... : regeneration times

[Rj, Rj+1) : regenerative cycle or period

• Cycle total (Lj) and cycle length (Nj) of cycle j

• Steady state mean

1

1

1,j

j

R

j i j j ji R

L X N R R

1 1

1 1

1

lim lim1

n n

j jj j

n nn n

j jj j

L LE L n

E NN N

n

47

Perturbation analysis of GSMP

48

Functional GSMP

System paramets :• Assume that the probability distribution of new clocks depend on a

continuous parameter .  Let :• Fe(x; ) : distribution of new clocks of event e

• X(e, n) : n-th clock of event e Hypothesis (H1) : • e and , Fe(x; ) is continuous in x and Fe(x; ) = 0. Hypothesis (H2) : • e and , X(e, n) is continuous differentiable in . Remark : • Under H1, the probability that two clocks reach zero at the same time is null

49

Perturbation generation

Remark : U = Fe(X(e, n); ) est une v.a. uniforme dans [0, 1]

The inverse transform technique for generation of event clocks X :1.Generate a random variable U uniformly distributed on [0, 1]2.X(e, n) = Fe

-1(U, )

Property : When the inverse tranform technique is used for generation of event clocks, the following infinitesimal perturbations are generated :

dX e, n F X e,n ; x F X e, n ;

d

50

Perturbation generation

Special cases :

• scale parameters : Fe(x; ) = Ge(x/)

dX(e, n) / d = X(e, n) /

Ex : a exponentially distributed random variable with mean  • location parameters : Fe(x; ) = Ge(x - )

dX(e, n) / d = 1

Ex : X = + L where is a location parameter

51

Perturbation propagation

Theorem: Under hypothese NI, H1 and H2, with probability 1, (i)the events en and the states sn remain unchanged within a small enough neighorhood of ;

(ii)the sojourn time n and event clocks Cn(e) are continuously differentiable at . Further,

  11 n nn

dC ed

d d

11

1

1 1

,

, , 1,

n n nn n

n

n n n

dC e dC ee E S e

dC e d dd dX e N e n

e E S E S ed

52

Sensitivity of performance measure

Theorem : The performance measures LT, LN et L, k are all differentiable in with

 dLT

d

dn

df Sn 1

n1

N T f SN T

d n

dn1

N T

dLN

d

d n

df Sn 1

n1

N

dL,k

d

d n

df Sn 1

n1

N T ,k

where N(t) is the number of events occurred in (0, t).

What about the average cost?

53

Simulation algorithm and perturbation analysis of an GSMP

1. Initialization :• Number of events : n = 0• Set initial state : S0• Set clocks of activated events : C0(e) = X(e, 1), e E(S� 0)

Gradient of C0(e) (perturbation generation):

• Set event counter : N(e, 0) = 0, e �2. Determine the next event :

3. Sojourn time at Sn : n+1 = Cn(en+1)Gradient of n+1 (perturbation propagation):

4. Update the event counters :

e n1 arg mine E s

C n e

11 n nndC ed

d d

1 ,1ndC e dX e

d d

54

5. Update the system state :Sn+1 = (Sn , en+1 , U(en+1, N(en+1, n+1)))

6. Update existing clocks :Cn+1(e) = Cn+1(e) - n+1, e E(sn) -{en+1}Gradient of Cn+1(e) (perturbation propagation):

7. Generate new clocks :Cn+1(e) = X(e, N(e, n) +1), e E(sn+1) - (E(sn) -{en+1})Gradient of Cn+1(e) (perturbation generation):

8. If the end of simulation, stop. Otherwise, n:= n+1 and go to step 2

1, , 1

ndX e N e ndC e

d d

1 1n n n ndC e dC e dC e

d d d

Simulation algorithm and perturbation analysis of an GSMP

55

Correctness of gradient estimators

56

We only check the unbiasedness, i.e.

A counter-example

where w is a uniform random variable on [0, 1]

Necessary and sufficient condition for (1) : uniform differentiability of L

Uniform differentiability is in general impossible to check directly.

Correctness of gradient estimators

1, if

0, if L

, (1)

dE L dLE

d d

57

Commuting Condition (CU)

CU condition: For all s1, s2, s3, , and such that

E(s1) and p(s2; s1, ) p(s3; s2, ) > 0 ,

there exist a state s4 such that:

p(s4; s1, b) = p(s3; s2, b) and p(s3; s4, a) = p(s2; s1, a).

Further, p(z1 ; s, ) = p(z2 ; s, ) > 0 z1 = z2.

s1

s2

s3

s4

Example: G/G/1 (CU), G/G/1/K (no CU)

58

Commuting Condition (CU)

Remarks :

•CU condition ensures the continuity of the state trajectory.

•The perturbation of two events only generates temporal perturbations of the state trajectory.

•The last part of the CU condition is not restrictive and we usually limit ourselves to the first part.

Example : CU condition holds for the inspection system example.

59

Commuting Condition (CU)

Hypothis (H3) : There exists a constant B > 0 such that:

Theorem CU : Under (NI), (H1) - (H3) and (CU).

•If E[sup N(T)2] < ∞ , then GSMP gradient estimator of LT is unbiased,

•If E[sup tN] < ∞, then GSMP gradient estimator of LN is unbiased,

•If E[sup N(T(, k))2] < ∞ and E[sup T(, k)2] < ∞, then GSMP gradient estimator of Lk is unbiased,

where N(t) is the number of event occurred in in [0, t] Remark :

•Conditions E[sup N(T)2] < ∞ , E[sup tN] < ∞, E[sup N(T(, k))2] and E[sup T(, k)2] < ∞ hold for most models of production systems.

•In general, we limit ourselves to the verification of other conditions. 

dX e, n d

B X e, n 1

60

Correctness conditions of Perturbation Analysis

Conditions on event clocks

Hypothesis (H1) : e and , Fe(x; ) is continuous in x and Fe(x; ) = 0.

Hypothesis (H2) : e and , X(e, n) is continuous differentiable in .

Hypothèse (H3) : There exists a constant B > 0 such that:

Conditions on the GSMP structureNI condition: s, s' �S and e E(s), p(s'; s, e) > 0 E(s')   E(s) - {e}

CU condition: For all s1, s2, s3, , and such that

E(s1) and p(s2; s1, ) p(s3; s2, ) > 0 ,

there exist a state s4 such that:

p(s4; s1, b) = p(s3; s2, b) and p(s3; s4, a) = p(s2; s1, a).

Further, p(z1 ; s, ) = p(z2 ; s, ) > 0 z1 = z2.

dX e, n d

B X e, n 1

61

Correctness conditions of Perturbation Analysis

A GSMP can be represented by its event diagram in which•nodes = states•arcs = possible transitions  Event diagrams are particular useful for checking NI and CU conditions.

s

s1

s2

s3 P1

P2

, 1

62

Applications to manufacturing systems

63

A. G/G/1 queue for departure time, throughput rate, average waiting time, ...), G/M/1, M/G/1, M/M/1

B. Optimization of mean WIP of production lines

C. Failure-prone continuous flow production lines (wrt failure/repair/production rates)

 

M1 B1 M2 B2 M3

64

D. KANBAN systems•Each Kanban stage is associated with a given number of kanbans•each product in a kanban stage is associated with a kanban•Performance measure: average product waiting time

M1 I1 O1 I2 O2 M2

Kanban stage 1 Kanban stage 2

65

E. A flexible manufacturing system•The system is composed of a set of machines (servers) and a given number of pallets (clients) for material handling•The path of a client through the system is a Markov chain with a probability qij to joining machine j after leaving i•The queues are of unlimited capacity•Performance measures: time for server 1 to serve N clients.

66

F. A flexible cell•A flexible machine produces sucessively for three dedicated machines (round robin rule)•Performance measures : mean WIP

M0 B0

B1 M1

B2 M2

B3 M3

67

Sensitivity estimation revisited

68

• Performance measure:

J() = E[L[X1(), X2(), ...]]

• Focus on scalar case for simpliciy, i.e. J() = E[L[X()]].

• Assume that X() has continuous probability distribution F(x, ).

• Then,

Problem setting

,J L x dF x

69

• Represent X as a continuous function of by inverse transformation X() = F-1(U,) where U = UNIF(0, 1)

• Then

• By differentiation (provided the interchange of expectation and differentiation is allowed)

• Further,

The IPA approach

1

0,J L x u du

1

0

,L x udJdu

d

, , ,L X U L X U X U

X

perturbation

generationperturbation propagation

70

•

• L(x) is used instead of L[x()], and a common realization of X is used while varies

• By differentiation (provided the interchange of expectation and differentiation is allowed)

Likelihood Ratio Approach (LR approach)

;J L x f x dx

;dJ f xL x dx

d

71

•

• Since

• if f(x; ) ≠0

Likelihood Ratio Approach (LR approach)

;dJ f xL x dx

d

ln ; ; 1

;

f x f x

f x

ln ; ln ;;

dJ f x f XL x f x dx E L X

d

ln ;f XL X

Likehood Ratio estimatorScore Function estimator

72

• IPA ≠ LR

• IPA needs more detailed system dynamics analysis• LR is easier to obtain

• IPA is more accurate which variance goes to zero as time horizon increase• LR has large variance which increases to infinity for the time horizon

increase

IPA or LR

73

Extensions of perturbation analysis

74

Ex 1 : A production line where the performance measure is the average number of parts inspected per time unit. Ex 2 : Inspection machine with an input buffer of limited capacity

State-dependent routing Ex 3 : Jackson network where q is a routing probability

Structural parameter  Discontinuity of performance measures (Ex 1)Discontinuity of the sample path (Ex 2 and 3)

When IPA does not work

75

Solution 1 : change of the representation of the stochastic processEx : Birth-death process Solution 2 : change of measures (case 1) For a production line, from Little's law,

L = Twhere Lis the average WIP, the throughput rate, T the mean sojourn time. Then,

and dL/d and dT/d can often be estimated by IPA

dd

dL d dT d

T

76

Solution 3 : Conditional perturbation analysis (SPA)

•The biase of an IPA estimator, i.e. E[dL/dq] - dE[L]/dq, is essentially due to the discontinuities of the random performance function L

•SPA consists in evaluating the derivative of a conditional performance function, , i.e.

d[E[L /z]]/d•The SPA estimator is unbiased if :

•In general, E[L /z] is better smoothed than L and SPA estimator is more likely to be unbiased.

EzdE L / z

d

dEz E L / z d