View
97
Download
0
Category
Tags:
Preview:
DESCRIPTION
Point Estimation. Notes of STAT 6205 by Dr. Fan. Overview. Section 6.1 Point estimation Maximum likelihood estimation Methods of moments Sufficient statistics Definition Exponential family Mean square error (how to choose an estimator). Big Picture. - PowerPoint PPT Presentation
Citation preview
Point Point EstimationEstimation
Notes of STAT 6205 by Dr. Fan
OverviewOverview• Section 6.1• Point estimation• Maximum likelihood estimation• Methods of moments• Sufficient statistics
o Definitiono Exponential familyo Mean square error (how to choose an estimator)
6205-Ch6 2
Big Picture• Goal: To study the unknown distribution of a
population• Method: Get a representative/random sample and
use the information obtained in the sample to make statistical inference on the unknown features of the distribution
• Statistical Inference has two parts: o Estimation (of parameters)o Hypothesis testing
• Estimation:o Point estimation: use a single value to estimate a parametero Interval estimation: find an interval covering the unknown parameter
6205-Ch6 3
Point EstimatorPoint Estimator• Biased/unbiased: an estimator is called unbiased
if its mean is equal to the parameter of estimate; otherwise, it is biased
• Example: X_bar is unbiased for estimating mu
6205-Ch6 4
Maximum Likelihood Estimation
• Given a random sample X1, X2, …, Xn from a distribution f(x; ) where is a (unknown) value in the parameter space
• Likelihood function vs. joint pdf
• Maximum Likelihood Estimator (m.l.e.) of , denoted as is the value which maximizes the likelihood function, given the sample X1, X2, …, Xn.
n
iixfxL
1
);();(
6205-Ch6 5
Examples/Exercises• Problem 1: To estimate p, the true probability of heads
up for a given coin.• Problem 2: Let X1, X2, …, Xn be a random sample
from a Exp(mu) distribution. Find the m.l.e. of mu.• Problem 3: Let X1, X2, …, Xn be a random sample
from a Weibull(a=3,b) distribution. Find the m.l.e. of b.
• Problem 4: Let X1, X2, …, Xn be a random sample from a N(,^2) distribution. Find the m.l.e. of and .
• Problem 5: Let X1, X2, …, Xn be a random sample from a Weibull(a,b) distribution. Find the m.l.e. of a and b.
6205-Ch6 6
Method of MomentsMethod of Moments• Idea: Set population moments = sample
moments and solve for parameters
• Formula: When the parameter is r-dimensional, solve the following equations for :
6205-Ch6 7
n
i
ki
k ,...r,knXXE1
21for /)(
Examples/ExercisesExamples/ExercisesGiven a random sample from a population
•Problem 1: Find the m.m.e. of for a Exp() population.
•Exercise 1: Find the m.m.e. of and for a N(^2) population.
6205-Ch6 8
Sufficient StatisticsSufficient Statistics• Idea: The “sufficient” statistic contains all
information about the unknown parameter; no other statistic can provide additional information as to the unknown parameter.
• If for any event A, P[A|Y=y] does not depend on the unknown parameter, then the statistic Y is called “sufficient” (for the unknown parameter).
• Any one-to-one mapping of a sufficient statistic Y is also sufficient.
• Sufficient statistics do not need to be estimators of the parameter.
6205-Ch6 9
Sufficient StatisticsSufficient Statistics
6205-Ch6 10
Examples/ExercisesExamples/ExercisesLet X1, X2, …, Xn be a random sample from f(x)
Problem: Let f be Poisson(a). Prove that1.X-bar is sufficient for the parameter a2.The m.l.e. of a is a function of the sufficient statistic
Exercise: Let f be Bin(n, p). Prove that X-bar is sufficient for p (n is known). Hint: find a sufficient statistic Y for p and then show that X-bar is a function of Y
6205-Ch6 11
Exponential FamilyExponential Family
6205-Ch6 12
Examples/ExercisesExamples/Exercises
Example 1: Find a sufficient statistic for p for Bin(n, p)
Example 2: Find a sufficient statistic for a for Poisson(a)
Exercise: Find a sufficient statistic for for Exp()
6205-Ch6 13
Joint Sufficient StatisticsJoint Sufficient Statistics
Example: Prove that X-bar and S^2 are joint sufficient statistics for and of N(, ^2)
6205-Ch6 14
Application of SufficienceApplication of Sufficience
6205-Ch6 15
ExampleExampleConsider a Weibull distribution with parameter(a=2, b)
1)Find a sufficient statistic for b
2)Find an unbiased estimator which is a function of the sufficient statistic found in 1)
6205-Ch6 16
Good Estimator?Good Estimator?• Criterion: mean square error
6205-Ch6 17
ExampleExample• Which of the following two estimator of variance
is better?
6205-Ch6 18
Recommended