Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
MultinomialGeometric
HypergeometricPoisson
Math 141Lecture 4: Distributions Related To The Binomial Distribution
Albyn Jones1
1Library [email protected]
www.people.reed.edu/∼jones/courses/141
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Outline
ReviewThe Multinomial DistributionThe Geometric and Negative Binomial DistributionsThe Hypergeometric DistributionThe Poisson Distribution
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Examples of Different Experiments
Binomial: Count the number of Heads in a fixed number oftosses.Geometric: Count the number of Tails before the first Head.Negative Binomial: Count the number of Tails beforebefore the k -th Head.Hypergeometric: Count the number of red cards dealt in apoker hand.Poisson A model for rare events.
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Review: The Binomial
Dichotomous Trials: Each trial results in either a ‘Success’,S, or a ‘Failure’ F .Independence: Successive trials are independent; knowingwe got S on one trial does not help us predict the outcomeof any other trial.Constant probability: Each trial has the same probabilityP(S) = p, and P(F ) = 1− p.X counts the number of S’s.n the number of trials is fixed.
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
X ∼ Binomial(n,p) Probabilities
Let p = P(S) and q = 1− p = P(F ), then
P(X = k) =
(nk
)pk qn−k
And in R, the density function is given by
P(X = k) = dbinom(k ,n,p)
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
The Multinomial
Polychotomous Trials: Each trial results in one of a fixedset of possible outcomes E1,E2, . . .EN. Example: dierolls, Ω = 1,2,3,4,5,6.Independence: Successive trials are independent; knowingthe outcome of one trial does not help us predict theoutcome of any other trial.Constant probability: Each trial has the same probabilityfor each possibility.X1,X2, . . .XN count the number of events of each type.n the number of trials is fixed.
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Multinomial Probabilities
Good news! The typical computation involves collapsingcategories to create a binomial.
Example: Roll a fair die 20 times. For each roll, there aresix possible outcomes, so we have a MultinomialDistribution.What is the probability of rolling three 6’s in 20 rolls?Let X be the number of 6’s in 20 rolls. What is thedistribution of X?
X ∼ Binomial(20,1/6)
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
But Since You Asked...
For n independent Multinomial(p1,p2, . . . ,pN ) trials, where theprobability of observing category i is pi . Let Xi be the count ofevents observed in category i , where
N∑i=1
Xi = n andN∑
i=1
pi = 1
P(X1 = k1, . . . ,XN = kN) =n!
k1!k2! . . . kN !pk1
1 pk22 . . . pkN
N
with R functions: rmultinom, dmultinom.
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Example: Election Polls
Suppose we ask registered Republicans for their preference:Bachman, Gingrich, Perry, Romney, or ‘none of the above’ (RonPaul is invisible :-).
The Poll We ask 1000 randomly chosen Republicans fortheir preference, and let 9,290,108,395,198 be thecounts for those 5 options, in order.The Population Proportions Suppose that the actualprobabilities are (in order) .01, .3, .1, .4, .19.The Probability:dmultinom(c(9, 290, 108, 395 , 198),1000,
c(.01, .3, .1, .4, .19))[1] 2.572792e-06
That looks small, but even the most likely outcome hassmall probability (about 5× 10−6)!
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
The Geometric Distribution
Dichotomous Trials: Each trial results in either a ‘Success’,S, or a ‘Failure’ F .Independence: Successive trials are independent; knowingwe got S on one trial does not help us predict the outcomeof any other trial.Constant probability: Each trial has the same probabilityP(S) = p, and P(F ) = 1− p = q.X counts the number of F ’s before the first S.Question: What is the probability we see k failures beforethe first success?
Pr(X = k) = P(F1,F2,F3, . . . ,Fk ,S) = p · qk
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Example
Roll a fair die repeatedly until getting the first 6.What are p and q here?What is the probability it comes on the 6th roll, that is wehave 5 non-sixes before the first 6?
P(X = 5) =
(56
)5 16≈ .067
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
The Negative Binomial Distribution
Like the Geometric: Dichotomous outcomes F ,S,independent trials, constant probability.X counts the number of F ’s before the r th S.Question: What is the probability we see k failures beforethe r th success?Hint The last trial must be an S, so we see k failures and(r − 1) successes (in any order), followed by a success.
Pr(X = k) =
(r − 1 + k
k
)·pr−1 ·qk ·p =
(r − 1 + k
k
)·qk ·pr
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Example
Roll a fair die repeatedly until getting the third 6.What are p and q here?What is the probability it comes on the 10th roll, that is wehave 7 non-sixes before the third 6?
P(X = 7) =
(97
)(56
)7(16
)3
≈ .0465
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Connections
The number of Failures observed before getting the r thSuccess is clearly the sum of
The number of Failures observed before the 1st SuccessThe number of Failures observed between the 1st and 2ndSuccessesThe number of Failures observed between the 2nd and 3rdSuccessesand so on.Theorem The sum of r independent Geometric(p) RV’s hasa NegativeBinomial(r ,p) distribution.
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
The Hypergeometric Distribution
Sampling from a finite population of two categories A and Bwithout replacement.
Non-Independence, Non-constant Probability: Successivetrials are dependent; every trial changes the sample spaceand probabilities for the subsequent trials.X counts the number of A’s.n the number of trials is fixed.
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Hypergeometric Probabilities
Suppose we have a bag with A alabaster and B black marbles,well mixed. We extract n marbles. Let X be the number ofalabaster marbles drawn. For 0 ≤ k ≤ min(A,n), the probabilityof drawing k alabaster and n − k black marbles is given by
P(X = k) =
(Ak
)( Bn−k
)(A+Bn
)
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Hypergeometric Example
Question: What is the probability that a 5 card poker handdealt from a well shuffled deck has 4 spades?There are A = 13 spades, B = 39 non-spades. Let X bethe number of spades dealt.
P(X = 4) =
(134
)(391
)(525
) ≈ 0.01
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
The Poisson DistributionA probability model for Rare Events
Origin: An analytical approximation for binomialprobabilities when n is large and p is small: let µ = np,then
P(X = k) ≈ µk
k !e−µ
Poisson Process A random process describing occuranceof events in time: let µ be the rate per unit time, then ifdisjoint time intervals are independent, and Xt counts thenumber of events occuring in an interval of length t ,
P(Xt = k) =(tµ)k
k !e−tµ
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Poisson Example
Question: What is the probability that in a group of 40people, no two share a birthday? Or, the complement: atleast two share a birthday?Poisson Approximation: what is the relevant Poissondistribution? To use Poisson approximation to theBinomial, we need to know how many ‘trials’ there are, andthe probability of success on each trial.Trials: How many trials are there here?(
402
)=
40!
2! · 38!=
40 · 392
= 780
Probabilities: The probability two people share a birthdayis approximately
1365
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Solution
The probability that no two of 40 people share a birthday isapproximately the probability that a Poisson RV X withparameter µ = 780/365 ≈ 2.137 is equal to 0.
P(X = 0) =µ0
0!e−µ = e−780/365 ≈ .12
Thus the probability that at least two people share a birthday isabout .88.
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Sums of Poisson(µ) RV’s
Suppose that X and Y are independent Poisson(µ) RV’s. Whatis the distribution of X + Y ?
Hint: if X and Y are independent Binomial(n,p) RV’s, thenX + Y is Binomial(n + n,p).Suppose that n is large, and p small, and µ = np.Conclusion?A Binomial(2n,p) is close to a Poisson(2µ)In General: the sum of independent Poisson(µi ) RV’s isPoisson(µ), where µ =
∑µi .
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Connecting the Poisson and Negative Binomial
Short version: The Negative Binomial is a good model for acollection of Poisson RV’s with variable rates µ.
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
R Functions
Distribution density CDF RNG
Binom(n,p) dbinom(k,n,p) pbinom(k,n,p) rbinom(N,n,p)
Geom(p) dgeom(k,p) pgeom(k,p) rgeom(N,p)
Hyper(A,B,n) dhyper(k,A,B,n) phyper(k,A,B,n) rhyper(N,A,B,n)
Poisson(µ) dpois(k,µ) ppois(k,µ) rpois(N,µ)
Albyn Jones Math 141
MultinomialGeometric
HypergeometricPoisson
Summary
The Multinomial DistributionThe Geometric and Negative Binomial DistributionsThe Hypergeometric Distribution: sampling withoutreplacement from a finite population.The Poisson Distribution: rare events.
Albyn Jones Math 141