View
6
Download
0
Category
Preview:
Citation preview
Limit Theorems
Limit TheoremsSTATISTICS – Lecture no. 6
Jirı Neubauer
Department of Econometrics FEM UO Brnooffice 69a, tel. 973 442029email:Jiri.Neubauer@unob.cz
3. 11. 2009
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
The Law of Large Numbers
If we repeat some experiment independently we can createusing given observed values distribution of relative frequenciesand calculate some measures (mean, median, variance . . . ).
This distribution (measures) we call sample distribution(sample measures).
Under particular conditions we can expect that the sampledistribution (measures) will converge toward a theoreticaldistribution (measures). The more repetitions of theexperiment the better convergence.
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
The Law of Large Numbers
If we repeat some experiment independently we can createusing given observed values distribution of relative frequenciesand calculate some measures (mean, median, variance . . . ).
This distribution (measures) we call sample distribution(sample measures).
Under particular conditions we can expect that the sampledistribution (measures) will converge toward a theoreticaldistribution (measures). The more repetitions of theexperiment the better convergence.
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
The Law of Large Numbers
If we repeat some experiment independently we can createusing given observed values distribution of relative frequenciesand calculate some measures (mean, median, variance . . . ).
This distribution (measures) we call sample distribution(sample measures).
Under particular conditions we can expect that the sampledistribution (measures) will converge toward a theoreticaldistribution (measures). The more repetitions of theexperiment the better convergence.
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
The Law of Large Numbers
Notice that the convergence of the sample values towardtheoretical ones is not the convergence in the sense ofmathematical convergence, but the probability convergence.
The probability convergence – if the number of experimentsincreases, the probability of deviation between sample valuesand theoretical values decreases.
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
The Law of Large Numbers
Notice that the convergence of the sample values towardtheoretical ones is not the convergence in the sense ofmathematical convergence, but the probability convergence.
The probability convergence – if the number of experimentsincreases, the probability of deviation between sample valuesand theoretical values decreases.
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Convergence in probability
Definition
If the sequence of random variables X1,X2, . . . ,Xn, . . . fulfils
limn→∞
P(|Xn − c | < ε) = 1, ε > 0,
it is said that the sequence {Xn} converges in probability to theconstant c , we write
XnP→ c .
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Chebyshev’s Inequality
Theorem
For any random variable X with the mean E (X ), the finitevariance D(X ) and for every ε > 0 we have
P(|X − E (X )| < ε) ≥ 1− D(X )
ε2.
Chebyshev’s inequality is useful fist of all in the theoretical field. Itallow us to estimate some probabilities of random variables withunknown distribution.
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Bernoulli’s Theorem
Theorem
If the random variable X denotes the number of occurrence of theevent in the sequence of n independent experiments, where π isthe probability of occurrence of the event in one experiment, thenfor every ε > 0 is
limn→∞
P
(∣∣∣∣Xn − π
∣∣∣∣ < ε
)= 1.
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
de Moivre-Laplace Theorem
Theorem
Let X be a random variable with binomial distributionX ∼ B(n, π) a For the standardized random variable
U =X − nπ√nπ(1− π)
we havelim
n→∞P(U ≤ u) = Φ(u),
where Φ(u) is the distribution function of the standard normaldistribution N(0, 1).
aX = X1, X2, . . . , Xn, where Xi , i = 1 . . . , n, are independent Bernoullirandom variables E(Xi ) = π, D(Xi ) = π(1− π), which means E(X ) = nπ andD(X ) = nπ(1− π).
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
de Moivre-Laplace Theorem
The de Moivre-Laplace theorem says that for n →∞ the binomialdistribution converges to the normal distribution.Given approximation is acceptable if
nπ(1− π) > 9 and1
n + 1< π <
n
n + 1.
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
de Moivre-Laplace Theorem for proportion
Theorem
Let X be a random variable with binomial distribution X ∼ B(n, π)The random variable X
n has the mean E(
Xn
)= π and the variance
D(
Xn
)= π(1−π)
n . For the standardized random variable
U =Xn − π√π(1− π)
√n
we havelim
n→∞P(U ≤ u) = Φ(u),
where Φ(u) is the distribution function of the standard normaldistribution N(0, 1).
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Levy-Lindeberg’s Theorem
Theorem
Let the random variable be X = X1 + X2 + · · ·+ Xn, whereXi , i = 1, . . . , n are independent random variables with the samedistribution with the mean E (Xi ) = µ and the finite varianceD(Xi ) = σ2, a For the standardized random variable
U =X − nµ√
nσ2
we havelim
n→∞P(U ≤ u) = Φ(u),
where Φ(u) is the distribution function of the standard normal
distribution N(0, 1).
aE(X ) = nµ and D(X ) = nσ2
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Levy-Lindeberg’s Theorem for the Mean
Theorem
Let the random variable X be the mean of n independent randomvariables X1,X2, . . . ,Xn, with the same distribution and the meanE (Xi ) = µ and the finite variance D(Xi ) = σ2, i = 1, . . . , n, then
E (X ) = µ and D(X ) =σ2
n
and for the standardized random variable
U =X − µ
σ
√n
we havelim
n→∞P(U ≤ u) = Φ(u),
where Φ(u) is the distribution function of the standard normal
distribution N(0, 1).Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Levy-Lindeberg’s Theorem
For M = X1 + · · ·+ Xn is:
M =n∑
i=1Xi ∼ as.N(nµ, nσ2),E (M) = nµ,D(M) = nσ2
U = M−E(M)√D(M)
= M−nµ√nσ2
∼ as.N(0, 1)
P(M ≤ m) = F (m) ≈ Φ(
m−nµ√nσ2
)P(−u1−α/2 < m−nµ√
nσ2< u1−α/2
)= 1− α
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Levy-Lindeberg’s Theorem
For M = X1 + · · ·+ Xn is:
M =n∑
i=1Xi ∼ as.N(nµ, nσ2),E (M) = nµ,D(M) = nσ2
U = M−E(M)√D(M)
= M−nµ√nσ2
∼ as.N(0, 1)
P(M ≤ m) = F (m) ≈ Φ(
m−nµ√nσ2
)P(−u1−α/2 < m−nµ√
nσ2< u1−α/2
)= 1− α
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Levy-Lindeberg’s Theorem
For M = X1 + · · ·+ Xn is:
M =n∑
i=1Xi ∼ as.N(nµ, nσ2),E (M) = nµ,D(M) = nσ2
U = M−E(M)√D(M)
= M−nµ√nσ2
∼ as.N(0, 1)
P(M ≤ m) = F (m) ≈ Φ(
m−nµ√nσ2
)
P(−u1−α/2 < m−nµ√
nσ2< u1−α/2
)= 1− α
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Levy-Lindeberg’s Theorem
For M = X1 + · · ·+ Xn is:
M =n∑
i=1Xi ∼ as.N(nµ, nσ2),E (M) = nµ,D(M) = nσ2
U = M−E(M)√D(M)
= M−nµ√nσ2
∼ as.N(0, 1)
P(M ≤ m) = F (m) ≈ Φ(
m−nµ√nσ2
)P(−u1−α/2 < m−nµ√
nσ2< u1−α/2
)= 1− α
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Levy-Lindeberg’s Theorem
For the sample mean X is:
X = 1n
n∑i=1
Xi ∼ as.N(µ, σ2
n ),E (M) = µ,D(M) = σ2
n
U = X−E(X )√D(X )
= X−µσ
√n ∼ as.N(0, 1)
P(X ≤ x) = F (x) ≈ Φ(
x−µσ
√n)
P(−u1−α/2 < x−µ
σ
√n < u1−α/2
)= 1− α
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Levy-Lindeberg’s Theorem
For the sample mean X is:
X = 1n
n∑i=1
Xi ∼ as.N(µ, σ2
n ),E (M) = µ,D(M) = σ2
n
U = X−E(X )√D(X )
= X−µσ
√n ∼ as.N(0, 1)
P(X ≤ x) = F (x) ≈ Φ(
x−µσ
√n)
P(−u1−α/2 < x−µ
σ
√n < u1−α/2
)= 1− α
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Levy-Lindeberg’s Theorem
For the sample mean X is:
X = 1n
n∑i=1
Xi ∼ as.N(µ, σ2
n ),E (M) = µ,D(M) = σ2
n
U = X−E(X )√D(X )
= X−µσ
√n ∼ as.N(0, 1)
P(X ≤ x) = F (x) ≈ Φ(
x−µσ
√n)
P(−u1−α/2 < x−µ
σ
√n < u1−α/2
)= 1− α
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Levy-Lindeberg’s Theorem
For the sample mean X is:
X = 1n
n∑i=1
Xi ∼ as.N(µ, σ2
n ),E (M) = µ,D(M) = σ2
n
U = X−E(X )√D(X )
= X−µσ
√n ∼ as.N(0, 1)
P(X ≤ x) = F (x) ≈ Φ(
x−µσ
√n)
P(−u1−α/2 < x−µ
σ
√n < u1−α/2
)= 1− α
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Continuity Correction
In the case of using the normal distribution as an approximation ofa distribution of a discrete random variable, it is recommended toapply so called continuity correction which improves thisapproximation.If we calculate P(X ≤ x) or P(X ≥ x) by normal approximation,we get undervalued results. On the contrary if we calculateP(X < x) or P(X > x) by normal approximation, we getovervalued results.
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Continuity Correction
Some examples of continuity correction:
before correction x < 3 x ≤ 3 x = 5 x ≥ 7 x > 7after correction x < 2.5 x < 3.5 4.5 < x < 5.5 x > 6.5 x > 7.5
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 1
The probability that you hit the target is 0.8. What is theprobability that the difference between the number of hits in thesequence of 200 shots and the mean of the this number will not belarge than 10?
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 1
The binomial distribution:
E (X ) = nπ = 200 · 0.8 = 160
D(X ) = nπ(1− π) = 200 · 0.8 · (1− 0.8) = 32
P(150 ≤ X ≤ 170) = p(150) + p(151) + · · ·+ p(170) =
=(200150
)0.8150 · 0.250 +
(200151
)0.8151 · 0.249 + · · ·+
+(200170
)0.8170 · 0.230 = 0.937
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 1
The binomial distribution:
E (X ) = nπ = 200 · 0.8 = 160
D(X ) = nπ(1− π) = 200 · 0.8 · (1− 0.8) = 32
P(150 ≤ X ≤ 170) = p(150) + p(151) + · · ·+ p(170) =
=(200150
)0.8150 · 0.250 +
(200151
)0.8151 · 0.249 + · · ·+
+(200170
)0.8170 · 0.230 = 0.937
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 1
de Moivre-Laplace theorem:
F (x) ≈ Φ
(x − nπ√nπ(1− π)
)
P(150 ≤ X ≤ 170) = F (170)− F (149) ≈ Φ(
170−160√32
)−
−Φ(
149−160√32
)= 0.936
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 1
de Moivre-Laplace theorem (with continuity correction):
F (x) ≈ Φ
(x − nπ√nπ(1− π)
)
P(150 ≤ X ≤ 170)≈P(149.5 < X < 170.5) = F (170.5)− F (149.5) =
= Φ(
170.5−160√32
)− Φ
(149.5−160√
32
)= 0.937
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 1
Chebyshev’s inequality:
P(|X − E (X )| < ε) ≥ 1− D(X )
ε2
E (X ) = nπ = 200 · 0.8 = 160
D(X ) = nπ(1− π) = 200 · 0.8 · (1− 0.8) = 32
P(|X − 160)| < 10) ≥ 1− 32
102= 0.68
P(|X − 160)| < 11) ≥ 1− 32
112= 0.736
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 1
Chebyshev’s inequality:
P(|X − E (X )| < ε) ≥ 1− D(X )
ε2
E (X ) = nπ = 200 · 0.8 = 160
D(X ) = nπ(1− π) = 200 · 0.8 · (1− 0.8) = 32
P(|X − 160)| < 10) ≥ 1− 32
102= 0.68
P(|X − 160)| < 11) ≥ 1− 32
112= 0.736
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 1
Chebyshev’s inequality:
P(|X − E (X )| < ε) ≥ 1− D(X )
ε2
E (X ) = nπ = 200 · 0.8 = 160
D(X ) = nπ(1− π) = 200 · 0.8 · (1− 0.8) = 32
P(|X − 160)| < 10) ≥ 1− 32
102= 0.68
P(|X − 160)| < 11) ≥ 1− 32
112= 0.736
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 1
Chebyshev’s inequality:
P(|X − E (X )| < ε) ≥ 1− D(X )
ε2
E (X ) = nπ = 200 · 0.8 = 160
D(X ) = nπ(1− π) = 200 · 0.8 · (1− 0.8) = 32
P(|X − 160)| < 10) ≥ 1− 32
102= 0.68
P(|X − 160)| < 11) ≥ 1− 32
112= 0.736
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 2
In some elections the coalition obtained 52 % of votes. What is theprobability that in the public opinion research of the size 2600respondents the opposition won?
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 2
X . . . the number of respondents who voted the opposition
X ∼ B(2600; 0.48)
E (X ) = nπ = 2600 · 0.48 = 1248
D(X ) = nπ(1− π) = 2600 · 0.48 · (1− 0.48) = 648.96
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 2
X . . . the number of respondents who voted the opposition
X ∼ B(2600; 0.48)
E (X ) = nπ = 2600 · 0.48 = 1248
D(X ) = nπ(1− π) = 2600 · 0.48 · (1− 0.48) = 648.96
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 2
P(X > 1300) = 1− P(X ≤ 1300) = 1− [p(0) + · · ·+ p(1300)] =
= 1−[(2600
0
)· 0.480 · 0.522600 + · · ·+
+(26001300
)· 0.481300 · 0.521300
]= 1− 0.98031 = 0.01969
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 2
de Moivre-Laplace theorem:
F (x) ≈ Φ
(x − nπ√nπ(1− π)
)
P(X > 1300) = 1− P(X ≤ 1300) = 1− F (1300) ≈≈ 1− Φ
(1300−1248√
648.96
)= 1− 0.97939 = 0.02061
Jirı Neubauer Limit Theorems
Limit TheoremsThe Law of Large Numbersde Moivre-Laplace TheoremLevy-Lindeberg’s Theorem
Example 2
de Moivre-Laplace theorem (with continuity correction):
F (x) ≈ Φ
(x − nπ√nπ(1− π)
)
P(X > 1300) = 1− P(X ≤ 1300) ≈ 1− P(X < 1300.5) =
= 1− Φ(
1300.5−1248√648.96
)=
= 1− 0.98034 = 0.01966
Jirı Neubauer Limit Theorems
Recommended