Markov ChainAndrew Wang

Yum

Probability0.2

0.8

0.3

0.7

Monte Carlo Simulation

Examples: What are the most commonly visited spots

in the game of monopoly?

Drunkard's Walk

Markov Chain

Each day, you choose to eat either grapes, cheese, or lettuce:

1. Choice today affects preferences tomorrow2. cheese => tomato (0.5) || lettuce (0.5)3. grapes => grapes (0.1) || cheese (0.4) ||

lettuce (0.5)4. lettuce => grapes (0.4) || cheese (0.6)

nom

Terminology

Absorbing

Expected Number

Probability

Linear System

How is this even related to CS?

Gaussian EliminationGaussianElimination[m_?MatrixQ, v_?VectorQ] :=Last /@ RowReduce[Flatten /@ Transpose[{m, v}]]

2x + y - z = 8-3x - y + 2z = -11-2x + y + 2z = -3

[ 2 1 -1 | 8 ] [ 1 (1/3) (-2/3) | (11/3) ][ -3 -1 2 | -11 ] ===> [ 0 1 2/5 | (13/5) ][ -2 1 2 | -3 ] [ 0 0 1 | -1 ]

Terminology

Row Echelon Form

Reduced Row Echelon Form

Gaussian Elimination

what on Earth?

Example problemsFlip a coin a bunch of times:Expected number of flips before getting 6 heads in a row?

Roll a 6 sided die a bunch of times:Expected number of rolls before getting 6 consecutive

identical values in a row?

POTW (medium) 30 points N (N<1000, N even) players sit around a table; the

game begins with two opposite players having one die each. On each turn, the two players with dice roll them.

If a player rolls a 1, he passes the die to his neighbour on the left; if he rolls a 6, he passes the die to his neighbour on the right; otherwise, he keeps the die for the next turn.

The game ends when one player has both dice after they have been rolled and passed; that player has then lost.

What is the expected number of turns the game lasts? Give your answer rounded to ten significant digits.

POTW (medium) 30 points

N (N<1000, N even) players sit in circle.Players at opposite sides start with 6-side diePlayers roll the dice at the same time:• 1 => pass die to the left• 6 => pass die to the right• 2,3,4,5 don't mean anythingFind the expected number of times until 1

player has both dice

POTW (hard) 50 points

An infinitely long random string of digits: p1p2p3p4p5p6p7p8p9p10p11p12p13p14p15...Every integer will occur as a substring in X at

some index Q Ex: X = 12501672... Q2501= 2 , Q67 = 6Given integer N ( N has <= 18 digits ),Find the expected value of QN

Hint: use Knuth-Morris-Pratt Pattern Matching

POTW (hard) 50 points, hints

It can be proven that the expected value is always integer (doesn't mean I know how)

For large N:built-in double is not precise enoughUse high precision decimals (BigDecimal) or

integers during gaussian elimination

Hint: Use gaussian elimination for small N then find pattern.