MARKOV ANALYSIS Andrei Markov, a Russian Mathematician developed the technique to describe the movement of gas in a closed container in 1940 In 1950s,

MARKOV ANALYSISMARKOV ANALYSIS Andrei Markov, a Russian Mathematician

developed the technique to describe the movement of gas in a closed container in 1940

In 1950s, management scientists began to recognize that the technique could be adapted to decision situations which fit the pattern Markov described

MARKOV ANALYSISMARKOV ANALYSIS

In the early 1960s, the procedure has been used to describe

Marketing Strategies Plant maintenance decisions Stock market movements Account Receivable Projections

MARKOV ANALYSISMARKOV ANALYSIS

Markov analysis is used to analyze the current state and movement of a variable to predict the future occurrences and movement of this variable by the use of presently known probabilities.

TERMS USEDTERMS USED Market share – The fraction of a population

that shops at a particular store or market. When expressed as a fraction, market

shares can be used in place of state probabilities.

TERMS USEDTERMS USED State probability – The probability of an

event occurring at a point in time, say, the probability that a person will purchase a

product at a given store during a given month.

TERMS USEDTERMS USED

Transition probability – The conditional probability that we will be in a future state given a current state.

The following are the characteristics of Markov The following are the characteristics of Markov process as applied in business.process as applied in business.

1.1. The The time structuretime structure. It is the period of time . It is the period of time when a set of outcomes occurswhen a set of outcomes occurs2. The 2. The state variablesstate variables. These are variables that . These are variables that

describe the condition, or state, of the describe the condition, or state, of the system at each trial or period of the process.system at each trial or period of the process.

3. At a specific point, often at the beginning or 3. At a specific point, often at the beginning or end of a time period in the process, the end of a time period in the process, the condition being monitored may changed condition being monitored may changed from one state to another. from one state to another.

Characteristics of Markov Characteristics of Markov ProcessProcess

4. The decision maker is often uncertain about 4. The decision maker is often uncertain about whether the state variable will change or whether the state variable will change or remain the same; and this uncertainty is remain the same; and this uncertainty is reflected in a set of transition probabilities. reflected in a set of transition probabilities.

5. At a given point in the structure of the 5. At a given point in the structure of the process, the set of transition probabilities process, the set of transition probabilities representing the uncertainty of moving to representing the uncertainty of moving to any future state depend only on the current any future state depend only on the current state.state.

Characteristics of Markov Characteristics of Markov ProcessProcess

The following are the assumptions of Markov The following are the assumptions of Markov analysis as applied in business.analysis as applied in business.

1.1. The transition probabilities for a given The transition probabilities for a given beginning state of the system equals one.beginning state of the system equals one.2. The probabilities apply to all system 2. The probabilities apply to all system participants.participants.3. The transition probabilities are constant 3. The transition probabilities are constant over time.over time.4. The states are independent over time.4. The states are independent over time.

Assumptions of Markov Assumptions of Markov AnalysisAnalysis

Matrix of Transition ProbabilitiesMatrix of Transition Probabilities

It is the likelihood that the system It is the likelihood that the system in a given current state will remain in in a given current state will remain in the same state or move to another the same state or move to another state in the next period. state in the next period.

It is a proportion of customers It is a proportion of customers which a company retains or loses from which a company retains or loses from period to period.period to period.


P =P =

mnP

mP

mP

nPPPn

PPP

21

22221

11211


MMn = M = Mn-1(P)(P)where:where:

MMnn = matrix of market share at period = matrix of market share at period nn

MMn-1= matrix of market share at period = matrix of market share at period n-1n-1P P = matrix of transition probabilities= matrix of transition probabilities

STEADY STATE (Equilibrium)

One of the major properties of Markov chains is that, in the long run, the process usually stabilizes.

A stabilized system is said to approach steady state or equilibrium when the system’s state probabilities have become independent of time.

STEADY STATE (Equilibrium)

One of the major properties of Markov chains is that, in the long run, the process usually stabilizes.

A stabilized system is said to approach steady state or equilibrium when the system’s state probabilities have become independent of time.


Example: Find the steady state probabilities,

85.015.0

2.08.0[Pp Sp] = [Pp Sp]

Market Share AnalysisMarket Share Analysis

A community has two gasoline service stations, A community has two gasoline service stations, Petron and Shell. The residents of the Petron and Shell. The residents of the community purchase gasoline at the two community purchase gasoline at the two stations on a monthly basis. The marketing stations on a monthly basis. The marketing department of the Petron Company surveyed a department of the Petron Company surveyed a number of residents and found that the number of residents and found that the customers were not totally loyal to either brand customers were not totally loyal to either brand of gasoline. Customers were willing to change of gasoline. Customers were willing to change service stations as a result of advertising , service stations as a result of advertising , service and other factors.service and other factors.


The marketing department found that if a The marketing department found that if a customer bought their gasoline from Petron in customer bought their gasoline from Petron in any given month, there was only a 0.60 any given month, there was only a 0.60 probability that the customers would buy from probability that the customers would buy from Petron the next month and a .40 probability that Petron the next month and a .40 probability that the customer would buy their gas from Shell the the customer would buy their gas from Shell the next month. Likewise, if a customer traded with next month. Likewise, if a customer traded with Shell in a given month, there was an .80 Shell in a given month, there was an .80 probability that the customer would purchase probability that the customer would purchase gasoline from Shell in the next month and a .20 gasoline from Shell in the next month and a .20 probability that the customer would purchase probability that the customer would purchase gasoline from Petron.gasoline from Petron.


Required:Required:1.1. Set-up the transition probability matrix for Set-up the transition probability matrix for

this problem.this problem.2.2. Determine the probability that the customer Determine the probability that the customer

will trade in a) Petron and b) Shell?will trade in a) Petron and b) Shell?3.3. Determine the steady-state probabilities and Determine the steady-state probabilities and 4.4. If there are 3,000 customers in the If there are 3,000 customers in the

community who purchase gasoline, community who purchase gasoline, determine the number of customers that determine the number of customers that each company can anticipate in the long runeach company can anticipate in the long run

Predicting Future Market Predicting Future Market SharesShares

Suppose in the canteen of a certain university which Suppose in the canteen of a certain university which offers three types of combo meals, A, B and C, on a given offers three types of combo meals, A, B and C, on a given day, 35% of the customers choose A, 30% choose B, and day, 35% of the customers choose A, 30% choose B, and 35% choose C. Based on the canteen staff observations, 35% choose C. Based on the canteen staff observations, those ordering A on one day, 40% will reorder A the next those ordering A on one day, 40% will reorder A the next day, while 25% will order B and 35% will order C. Of those day, while 25% will order B and 35% will order C. Of those who order B on a day, 30% will reorder B the next day, who order B on a day, 30% will reorder B the next day, 40% will order A and 30% will order C. Of those who order 40% will order A and 30% will order C. Of those who order C on a day, 50% will reorder C the next day, 25% will C on a day, 50% will reorder C the next day, 25% will order A and 25% will order B. The transition probabilities order A and 25% will order B. The transition probabilities of the problem can be illustrated using a tree diagram.of the problem can be illustrated using a tree diagram.

Plant Maintenance SchedulePlant Maintenance Schedule

A particular production machine could be A particular production machine could be assigned the states “operating” and assigned the states “operating” and “breakdown”. The transition probabilities “breakdown”. The transition probabilities could then reflect the probability of a could then reflect the probability of a machine’s either breaking down or machine’s either breaking down or operating in the next time period. (i.e., operating in the next time period. (i.e., month, day, or year)month, day, or year)


As an example, consider a machine As an example, consider a machine having the following daily transition having the following daily transition matrix.matrix.

Day 1 Day 2

Operate Breakdown

T= Operate .90 .10

Breakdown .70 .30


The steady-state probabilities for this example The steady-state probabilities for this example areare.88 = steady-state probabilities for this example .88 = steady-state probabilities for this example areare.12 = steady-state probabilities of the machine’s .12 = steady-state probabilities of the machine’s breaking downbreaking down

If management decides that the long –run If management decides that the long –run probability of .12 for a breakdown is excessive, it probability of .12 for a breakdown is excessive, it might consider increasing preventive might consider increasing preventive maintenance, which would change the transition maintenance, which would change the transition matrix for this example.matrix for this example.


The marketing department found that if a The marketing department found that if a customer bought their gasoline from Petron in customer bought their gasoline from Petron in any given month, there was only a 0.60 any given month, there was only a 0.60 probability that the customers would buy from probability that the customers would buy from Petron the next month and a .40 probability that Petron the next month and a .40 probability that the customer would buy their gas from Shell the the customer would buy their gas from Shell the next month. Likewise, if a customer traded with next month. Likewise, if a customer traded with Shell in a given month, there was an .80 Shell in a given month, there was an .80 probability that the customer would purchase probability that the customer would purchase gasoline from Shell in the next month and a .20 gasoline from Shell in the next month and a .20 probability that the customer would purchase probability that the customer would purchase gasoline from Petron.gasoline from Petron.


Required:Required:1.1. Set-up the transition probability matrix for Set-up the transition probability matrix for

this problem.this problem.2.2. Determine the probability that the customer Determine the probability that the customer

will trade in a) Petron and b) Shell?will trade in a) Petron and b) Shell?3.3. Determine the steady-state probabilities and Determine the steady-state probabilities and

indicate the number of customers that each indicate the number of customers that each company can anticipate in the long run.company can anticipate in the long run.

EXAMPLE #1EXAMPLE #1 For the past twelve months, a computer For the past twelve months, a computer printer functioned 80% of the time printer functioned 80% of the time correctly during the current month if it correctly during the current month if it had functioned correctly in the preceding had functioned correctly in the preceding month. This means that 20% of the time month. This means that 20% of the time the printer did not function correctly for a the printer did not function correctly for a given month when it was functioning given month when it was functioning correctly during the preceding month. correctly during the preceding month.

EXAMPLE #1EXAMPLE #1 Moreover, it was observed that 85% of Moreover, it was observed that 85% of the time the printer remained the time the printer remained incorrectly adjusted for any given incorrectly adjusted for any given month if it was incorrectly adjusted the month if it was incorrectly adjusted the preceding month and 15% of the time preceding month and 15% of the time the printer operated correctly in a given the printer operated correctly in a given month when it did not operate correctly month when it did not operate correctly during the preceding month. during the preceding month.

EXAMPLE #1EXAMPLE #1a. What is the probability that the a. What is the probability that the printer will be functioning correctly one printer will be functioning correctly one month from now?month from now?

b. What is the probability that the b. What is the probability that the printer will be functioning correctly two printer will be functioning correctly two months months

from now?from now?

APPLICATIONSAPPLICATIONSThe mathematical function that describes this The mathematical function that describes this distribution, also called the density function, is given by distribution, also called the density function, is given by

where:where:X – is the value of the random variable X – is the value of the random variable - standard deviation of the distribution- standard deviation of the distribution- mean of the distribution- mean of the distribution= 2.14= 2.14e = 2.718e = 2.718

2

2

2

)(

2

1)(

X

ef

Documents

MARKOV ANALYSIS Andrei Markov, a Russian Mathematician developed the technique to describe the movement of gas in a closed container in 1940 In 1950s,