Chapter 1 - Introduction and Review of Probability Theory

2015 – 2016 (Sem 2) Ch 1 YK Chung (SAAS , HKU ) STAT2803/3903 Stochastic Models

Chapter 1:

Introduction and Review of Probability Theory

Department of Statistics and Actuarial Science

The University of Hong Kong

STAT 2803 / 3903

Stochastic Models

2015-2016 (2nd Semester)

1 / 55


1 Overview of The Course: Stochastic Process

Overview of the Course: Stochastic Process

Outline

2 Probability and Random Variables

3 Independence

4 Conditional Probability and Conditional Expectation

5 First Step Analysis

6 Probability Generating Function

2 / 55


Stochastic Process

A collection of random variables, indexed by an ordered subset of real numbers:

Overview of The Course: Stochastic Process Overview of The Course: Stochastic Process

• The set S of all possible values that the random variables X(t) can take is called the state space of the process.

• The index t is often, but not necessarily, interpreted as time.

• The index set T of the process, which is an ordered subset of real numbers, can be discrete or continuous.

• For each t T, X(t) is a random variable that follows a particular distribution.

• Stochastic processes are used to model the evolution of physical processes through time. The random variable X(t) is the random state of the process at time t.

• Each realization of the stochastic process, x(t), is a function of t. Therefore a stochastic process can be viewed as a random function of time t.

3 / 55


Examples of Stochastic Processes

• Xn = accumulated total points after rolling a dice n times, n = 1, 2, …


discrete time T = {1, 2, …}, discrete state space S = {1, 2, …} stochastic process

Markov Chain

4 / 55


Examples of Stochastic Processes

• X(t) = total number of customers visited a supermarket by time t


continuous time T , discrete state space S = {0, 1, 2, …} stochastic process

Poisson Process

• X(t) = closing price of a stock at the end of day t

discrete time T = {1, 2, …}, continuous state space S (0, ) stochastic process

Time Series

• X(t) = x-coordinate of the 3D location of a gas molecule at time t

continuous time T , continuous state space S stochastic process

Brownian Motion

5 / 55


• Required knowledge in mathematics

Matrix algebra

Calculus – univariate and multivariable

Differential equation (better, but not a must)

Setting the Stage Overview of The Course: Stochastic Process

• Required knowledge in probability theory

Conditional probability and conditional expectation

Distributions: Binomial, Poisson, Exponential, Gamma, Normal

Moment generating function

• Pre-study suggestion

Do some revision on STAT1801/2901 course materials

Read Chapter 1-3 of the text book (Introduction to Probability Models, by Sheldon M. Ross.)

6 / 55


Outline

Probability and Random Variables

7 / 55



3 Independence





• Probability defined on events

Probability Probability and Random Variables

For each event E of the sample space , the probability of event E is defined as a set function P(E), that satisfies the following conditions:

1

2

3 (Countable Additivity) For any sequence of events E1, E2, … that are mutually exclusive (En Em = when n m), then

8 / 55


• Properties of probability

Impossible event

Bounded

Non-occurrence

Implication relation

Boole’s inequality


9 / 55


• Inclusion-Exclusion Principle


• Allows us to calculate probability of union from probabilities of intersections.

10 / 55


• Example: An urn contains 2n balls labelled 1 to n in pairs. If we keep drawing balls two at a time without replacement. What is the probability that at the end, at least one matched pair can be drawn?


1

1

2

2

n

n

?

?

Let Ei = i th drawn pair matches

Probability of drawing at least one pair:

11 / 55


• Random Variable

a real-valued function of the outcomes in performing an experiment, i.e. defined on the sample space .

X is discrete if it takes on countable number of possible values.

X is continuous if it takes on a continuum (e.g. interval) of possible values.

Random Variable Probability and Random Variables

• Cumulative distribution function (cdf)

Non-decreasing

Right continuous

Limits

12 / 55


• If a random variable X takes on the values x1 , x2 , …, then the probability mass function (pmf) of X is defined as

Discrete Random Variable Probability and Random Variables

• Cumulative distribution function

• Expected value

13 / 55


• For continuous random variable X, the probability density function (pdf) of X is a nonnegative function defined on (– , ), such that for any set B of real numbers,

Continuous Random Variable Probability and Random Variables

• Cumulative distribution function

• Expected value

14 / 55


• The long-term average of a random variable X is represented by its (population) mean:

Mean, Variance, and Moments Probability and Random Variables

• The spread/variation of a random variable X is represented by its (population) variance:

• Formulae under linear transformation

• Other behaviors of a random variables are described by moments:

15 / 55


• A useful alternative specification of a random variable X is the moment generating function (mgf):

Moment Generating Function Probability and Random Variables

• Generate moments:

• The mgf, if exists, uniquely characterizes the distribution.

• Formulae under linear transformation

16 / 55


• A Bernoulli random variable X takes only values 0 or 1. The pmf is simply

where p is often called the success probability.

Binomial Distribution Probability and Random Variables

Number of ‘successes’ out of n independent Bernoulli trials (with the same p).

Sum of n independent Bernoulli random variables (with the same p).

Notation:

• A binomial random variable X takes values from {0, 1, …, n}, with pmf

17 / 55


Binomial Distribution Probability and Random Variables

• Moment generating function

• Mean and Variance

18 / 55


Poisson Distribution Probability and Random Variables

Common model of random counts, e.g. number of accidents, defects, injuries, insurance claims, …, etc.

Notation:

Approximation to binomial distribution:

• A Poisson random variable X takes values from {0, 1, 2, …}, with pmf

where > 0 is the parameter.

19 / 55


Poisson Distribution Probability and Random Variables



20 / 55


Exponential Distribution Probability and Random Variables

Common model of waiting time, e.g. lifetime, failure time, decay time of radioactive particle, interarrival times of insurance claims, …, etc.

Notation:

Important distribution for studying Poisson process and continuous time Markov chain.

Memoryless property:

• An Exponential random variable X takes values from (0, ), with pdf and cdf

where > 0 is the parameter.

21 / 55


Exponential Distribution Probability and Random Variables



• Moments

22 / 55


Gamma Distribution Probability and Random Variables

Common model of nonnegative random quantities positively skewed, e.g. size of insurance claims, rainfalls, accumulative waiting times, …, etc.

Notation:

Closely related with the exponential distribution:

Exponential is a special case of gamma with = 1.

Sum of independent and identically distributed exponential random variables is distributed as gamma.

• A Gamma random variable X takes values from (0, ), with pdf

where > 0, > 0 are the parameters, and () is the Gamma function with properties , , and .

23 / 55


Gamma Distribution Probability and Random Variables



24 / 55


Normal Distribution Probability and Random Variables

Common model of naturally occurring variables with symmetric random noises, e.g. heights, weights, blood pressures of adult humans, measurement errors, rates of change of logarithm of stock market indices, …, etc.

Notation:

Limiting distribution of sample mean, by central limit theorem (CLT).

Key component of the Brownian motion.

• A Normal (Gaussian) random variable X takes values from , with pdf

where – < < , > 0 are the parameters.

25 / 55


Normal Distribution Probability and Random Variables



• Linear transformation

• Standardization

26 / 55


Jointly Distributed Random Variables Probability and Random Variables

• Random Vector

vector-valued function that maps the outcomes in to p-dimensional vectors in p.

X is discrete if it takes on countable number of possible values.

X is continuous if it takes on a continuum (e.g. intervals) of possible values.

• Joint cdf

• Joint pmf (discrete)

• Joint pdf (continuous)

27 / 55


Jointly Distributed Random Variables Probability and Random Variables

• Marginal pmf (discrete)

• Marginal pdf (continuous)

• Expected value

28 / 55


Outline

Independence

29 / 55



3 Independence





Independence Independence

• Independence of events

The events E1, E2, …, En are said to be (mutually) independent if for every subset Ei1

, Ei2, …, Eir

(i1 < i2 < <ir, 2 r n) of these events,

Meaning: information (of the occurrences) on any subset of events tells us nothing about the others.

Important remark: pairwise independence

does not imply mutual independence.

30 / 55



• Independence of random variables

The random variables X1, X2, …, Xn are said to be independent if

for any real numbers x1, x2, …, xn.

Meaning: information on any subset of the random variables tells us nothing about the others.

Note that it implies the mutual independence of the events

for any sets A1, A2, …, An of real numbers.

31 / 55



• Independence of random variables

The random variables X1, X2, …, Xn are independent if and only if

or

for any real numbers x1, x2, …, xn.

(discrete case)

(continuous case)

• If the random variables X1, X2, …, Xn are independent, then X1, X2, …, Xk

are independent for any integer k such that 2 k n – 1.

• If the random variables X1, X2, …, Xn are independent, then g(X1, X2, …, Xk)

and h(Xk+1, Xk+2, …, Xn) are independent for any real-valued functions g and h, and integer k such that 2 k n – 1.

32 / 55


Outline

Conditional Probability and Conditional Expectation

33 / 55



3 Independence





Conditional Probability Conditional Probability and Conditional Expectation

• Suppose A and B are two events such that P(B)>0, the conditional probability that A occurs given that B has occurred is defined by

It allows us to access the uncertainty of event A, based on additional information from event B.

• If the events A and B are independent, then

That is, knowledge of occurrence of one event does not affect the probability of occurrence of the other event.

34 / 55


Law of Total Probability Conditional Probability and Conditional Expectation

• Law of Total Probability

Suppose B1, B2, …, Bn are mutually exclusive ( ) and exhaustive ( ) events, then for any event A,

It relates the marginal probabilities to conditional probabilities.

It can also be stated for conditional probabilities:

35 / 55


• Conditional Independence

Suppose A, B, C are events such that P(C) > 0. If

then A and B are said to be conditionally independent given C.

A restrictive form of independence between A and B under the condition that C has occurred.

Conditional Independence Conditional Probability and Conditional Expectation

• An important concept to understand the Markov property, which is essential in studying stochastic processes.

• Markov property: the future state and the past history are conditionally independent, given the current state.

36 / 55


Conditional Independence Conditional Probability and Conditional Expectation

• Example: There are two coins in a bag, one is unbiased and the other one is biased with Head probability 0.9. A coin is randomly drawn and then flipped twice independently. Consider the events

A : 1st flip results in a Head B : 2nd flip results in a Head F : the selected coin is biased

• Obviously A and B are conditionally independent, given F (or F c).

• Hence A and B are not independent when no information of F is obtained. In other words, A depends on B (or vice versa) through F.

• Conditional independence does not imply unconditional independence, and vice versa.

37 / 55


Bayes’ Formula Conditional Probability and Conditional Expectation

• Bayes’ Formula

Suppose A and B are events such that P(A) > 0, P(B) > 0, then

It enable us to evaluate the ‘inverse probability’ (conditional probability with events in reversed order).

It suggests a way to adjust the prior probability P(B) into the posterior probability P(B|A) after observed information from A.

It is often used together with the law of total probability:

38 / 55


Bayes’ Formula Conditional Probability and Conditional Expectation

• Example: There are two coins in a bag, one is unbiased and the other one is biased with Head probability 0.9. A coin is randomly drawn and then flipped twice independently. Consider the events

A : 1st flip results in a Head B : 2nd flip results in a Head F : the selected coin is biased

• The coin is more likely to be biased (76.4%) if both flips resulted in Heads.

39 / 55


Conditional Expectation Conditional Probability and Conditional Expectation

• For two random variables X and Y, the conditional distribution of X given Y = y is the distribution of X for the sub-population corresponding to the constraint Y = y.

• (Discrete case) Conditional pmf

• (Discrete case) Conditional pdf

• Conditional Expectation: expected value evaluated from sub-population:

• Note that h(y) = E(g(X)|Y = y) is a function of y. Therefore h(Y) = E(g(X)|Y) is a function of Y and hence a random variable.

40 / 55



• Conditional Mean

• Conditional Variance

• Properties

Linear transformation

If X and Y are independent, then

41 / 55



• Law of Total Expectation

It enables us to calculate expectations iteratively.

The law of total probability is a special case.

42 / 55



• Law of Total Variance / Variance Decomposition

It enables us to calculate variances iteratively.

The expectation E(Var(X|Y)) can be regarded as a measure of the within-group variation of X in the sup-populations defined by Y.

The variance Var(E(X|Y)) can be regarded as a measure of the between-group variation of X among the sup-populations defined by Y.

The formula can be interpreted as a decomposition of variation into components that explains the within-group and between-group variations.

43 / 55



• Example: The number of claims, N, on a block of insurance policies in a year follows (). Given N = n, the claim sizes (in $1000) are distributed as Gamma(9, 0.06) independently. What is the mean and variance of the total claim amount in a year?

Number of claims:

Claim sizes:

conditionally independent

Total claim amount: random sum

44 / 55



Number of claims:

Claim sizes:

(conditionally independent)

45 / 55


Outline

First Step Analysis

46 / 55



3 Independence





First Step Analysis First Step Analysis

• The first step analysis (FSA) is a useful technique for solving probability problems with hierarchical structures and Markov property.

Original problem

Intermediate stage

Case 1

Case 2

Case k

Result 1

Result 2

• Set up recursion equation(s) and solve.

47 / 55


First Step Analysis

• Example: Suppose the number of a man’s sons follows a (1.5) distribution, and the sons will independently have number of son’s following the same distribution, and so on. What is the probability that their surnames will eventually die out?

First step analysis: consider the first generation of spread

E : event that their surnames will eventually die out

Solving by Newton’s method:

• A special case of Branching Process, known as the Galton-Watson Process.

First Step Analysis

48 / 55


First Step Analysis

• Example: A biased coin with Head probability p is repeatedly flipped independently, until k consecutive Heads are obtained. What is the mean number of flips?

= Number of flips to obtain k consecutive Heads

First Step Analysis : consider the first time when (k – 1) consecutive Heads are obtained, i.e. the coin was flipped times, then

if the next flip is Head, k consecutive Heads are obtained;

If the next flip is Tail, we restart.

First Step Analysis

49 / 55


Outline

Probability Generating Function

50 / 55



3 Independence






• Let X be a nonnegative and integer-valued random variable. Then the probability generating function (pgf) of X is defined by


• Generate probabilities:

• Like mgf, the pgf uniquely characterizes the distribution.

• Relationship with the mgf:

51 / 55



• Example: Binomial distribution


• Example: Poisson distribution

• Example: Sum of Independent Random Variables

52 / 55



• Example: Random Sum where Xi are i.i.d. given N


Therefore the pgf of the random sum is the composite function of the pgfs of N and X, under the iid assumption.

53 / 55


Probability Generating Function Probability Generating Function

• The probability generating function is useful in solving differential equations and recurrence equations, which are often encountered in studying stochastic processes.

• Example: Suppose a stochastic process has pmfs satisfying

What is the distribution of X(t) ?

54 / 55


Probability Generating Function Probability Generating Function

• To solve the differential equation, we can multiply both side by and obtain an equation in terms of probability generating function:

• Hence which is the Poisson Process.

55 / 55

Documents

Chapter 1 - Introduction and Review of Probability Theory