Chapter 8

Continuous Time Markov Chains

Markov Availability Model

2-State Markov Availability Model

1) Steady-state balance equations for each state:– Rate of flow IN = rate of flow OUT

• State1:• State0:

2 unknowns, 2 equations, but there is only one independent equation.

UP1

DN0

MTTR

MTTF

1

1

10 01

2-State Markov Availability Model(Continued)

1) Need an additional equation: 110

Downtime in minutes per year = * 8760*60

1

11 111

min356.510199999.0 5 DTMYAA ssss

MTTRMTTF

MTTR

MTTRMTTF

MTTFAss

11

1

1

11

MTTRMTTF

MTTRAss

1

2-State Markov Availability Model(Continued)

2) Transient Availability for each state:– Rate of buildup = rate of flow IN - rate of flow OUT

This equation can be solved to obtain assuming P1(0)=1

)()( 101 tPtP

dt

dP

havewetPtP 1)()(since 10 )())(1( 111 tPtP

dt

dP

)()( 11 tP

dt

dP

tetAtP )(1 )()(

2-State Markov Availability Model(Continued)

3)

4) Steady State Availability:

tetR )(

ss

tAtA )(lim

• Assume we have a two-component parallel

redundant system with repair rate .

• Assume that the failure rate of both the components

is .

• When both the components have failed, the system

is considered to have failed.

Markov availability model

Markov availability model (Continued)

• Let the number of properly functioning components be the state

of the system. The state space is {0,1,2} where 0 is the system

down state.

• We wish to examine effects of shared vs. non-shared repair.

2 1 0

2

2

2 1 0

2

Non-shared (independent) repair

Shared repair

Markov availability model (Continued)

• Note: Non-shared case can be modeled & solved using a RBD

or a FTREE but shared case needs the use of Markov chains.

Markov availability model (Continued)

Steady-state balance equations

• For any state:Rate of flow in = Rate of flow outConsider the shared case

i: steady state probability that system is in state i

122 021 2)(

01

Steady-state balance equations (Continued)

• Hence

Since

We have

or

12 2

1210

01

12 000

2

20

21

1

Steady-state balance equations (Continued)

• Steady-state unavailability = 0= 1 - Ashared

Similarly for non-shared case,

steady-state unavailability = 1 - Anon-shared

• Downtime in minutes per year = (1 - A)* 8760*60

2

221

11

sharednonA

Steady-state balance equations

Homework 5:

• Return to the 2 control and 3 voice channels example and

assume that the control channel failure rate is c, voice channel

failure rate is v.

• Repair rates are c and v, respectively. Assuming a single

shared repair facility and control channel having preemptive

repair priority over voice channels, draw the state diagram of a

Markov availability model. Using SHARPE GUI, solve the

Markov chain for steady-state and instantaneous availability.

Markov Reliability Model

• Consider the 2-component parallel system but disallow repair

from system down state

• Note that state 0 is now an absorbing state. The state diagram

is given in the following figure.

• This reliability model with repair cannot be modeled using a

reliability block diagram or a fault tree. We need to resort to

Markov chains.

(This is a form of dependency since in order to repair a

component you need to know the status of the other

component).

Markov reliability model with repair

• Markov chain has an absorbing state. In the steady-state, system will be in state 0 with probability 1. Hence transient analysis is of interest. States 1 and 2 are transient states.

Markov reliability model with repair (Continued)

Absorbing state

Assume that the initial state of the Markov chain

is 2, that is, P2(0) = 1, Pk (0) = 0 for k = 0, 1.

Then the system of differential Equations is written

based on:

rate of buildup = rate of flow in - rate of flow out

for each state

Markov reliability model with repair (Continued)

Markov reliability model with repair (Continued)

)()()(2)(

121 tPtPdt

tdP

)()(2)(

122 tPtPdt

tdP

)()(

10 tPdt

tdP

After solving these equations, we get

R(t) = P2(t) +P1(t)

Recalling that , we get:

Markov reliability model with repair (Continued)

0

)( dttRMTTF

222

3

MTTF

Note that the MTTF of the two component parallel redundant system, in the absence

of a repair facility (i.e., = 0), would have

been equal to the first term,

3 / ( 2* ), in the above expression.

Therefore, the effect of a repair facility is to

increase the mean life by / (2*2), or by a

factor

Markov reliability model with repair (Continued)

13

)

2321(

2

Markov Reliability Model with Imperfect Coverage

Markov model with imperfect coverage

Next consider a modification of the above example proposed by Arnold as a model of duplex processors of an electronic switching system. We assume that not all faults are recoverable and that c is the coverage factor which denotes theconditional probability that the system recovers given that a fault has occurred. The state diagram is now given by the following picture:

Now allow for Imperfect coverage

c

Markov modelwith imperfect coverage (Continued)

Assume that the initial state is 2 so that:

Then the system of differential equations are:

0)0()0(,1)0( 102 PPP

)()()1(2)(

)()()(2)(

)()()1(2)(2)(

120

121

1222

tPtPcdt

tdP

tPtcPdt

tdP

tPtPctcPdt

tdP

Markov model with imperfect coverage (Continued)

After solving the differential equations we obtain:

R(t)=P2(t) + P1(t)

From R(t), we can system MTTF:

It should be clear that the system MTTF and system reliability are

critically dependent on the coverage factor.

)]1([2

)21(

c

cMTTF

SOURCES OF COVERAGE DATA

• Measurement Data from an Operational system: Large

amount of data needed;

Improved Instrumentation Needed

• Fault/Error Injection Experiments

Costly yet badly needed: tools from

CMU, Illinois, Toulouse

SOURCES OF COVERAGE DATA (Continued)

• A Fault/Error Handling Submodel

Phases of FEHM:

Detection, Location, Retry, Reconfig, Reboot

Estimate Duration & Prob. of success of each phase

IBM(EDFI), HARP(FEHM), Draper(FDIR)

Homework 6:

• Modify the Markov model with imperfect coverage to allow

for finite time to detect as well as imperfect detection. You

will need to add an extra state, say D. The rate at which

detection occurs is . Draw the state diagram and using

SHARPE GUI investigate the effects of detection delay on

system reliability and mean time to failure.