View
42
Download
0
Category
Tags:
Preview:
DESCRIPTION
STATISTICAL INFERENCE PART II SOME PROPERTIES OF ESTIMATORS. SOME PROPERTIES OF ESTIMATORS. θ: a parameter of interest; unknown Previously, we found good(?) estimator(s) for θ or its function g(θ). Goal: Check how good are these estimator(s). Or are they good at all? - PowerPoint PPT Presentation
Citation preview
1
STATISTICAL INFERENCEPART II
SOME PROPERTIES OF ESTIMATORS
SOME PROPERTIES OF ESTIMATORS
• θ: a parameter of interest; unknown
• Previously, we found good(?) estimator(s) for θ or its function g(θ).
• Goal:
• Check how good are these estimator(s). Or are they good at all?
• If more than one good estimator is available, which one is better?
2
3
SOME PROPERTIES OF ESTIMATORS
• UNBIASED ESTIMATOR (UE): An estimator is an UE of the unknown parameter , if
ˆE for all
Otherwise, it is a Biased Estimator of .
ˆ ˆBias E
Bias of for estimating
If is UE of , ˆ 0.Bias
4
SOME PROPERTIES OF ESTIMATORS
• ASYMPTOTICALLY UNBIASED ESTIMATOR (AUE): An estimator is an AUE of the unknown parameter , if
ˆ ˆ0 lim 0n
Bias but Bias
5
SOME PROPERTIES OF ESTIMATORS
• CONSISTENT ESTIMATOR (CE): An estimator which converges in probability to an unknown parameter for all is called a CE of .
ˆ .p
• MLEs are generally CEs.
For large n, a CE tends to be closer to the unknown population parameter.
6
EXAMPLE
For a r.s. of size n,
E X X is an UE of . By WLLN,
pX
X is a CE of .
7
MEAN SQUARED ERROR (MSE)
• The Mean Square Error (MSE) of an estimator for estimating is
22ˆ ˆ ˆ ˆMSE E Var Bias
If is smaller, is a better estimator of .
ˆMSE
1 2ˆ ˆ ,For two estimators, and of if
1 2ˆ ˆ ,MSE MSE
1 is better estimator of .
8
MEAN SQUARED ERROR CONSISTENCY
• Tn is called mean squared error consistent (or consistent in quadratic mean) if E{Tn}20 as n.
Theorem: Tn is consistent in MSE iff
i) Var(Tn)0 as n.
) lim .nnii E T
• If E{Tn}20 as n, Tn is also a CE of .
9
EXAMPLES
X~Exp(), >0. For a r.s of size n, consider the following estimators of , and discuss their bias and consistency.
Which estimator is better?
1n
X
T,n
X
T
n
1ii
2
n
1ii
1
10
SUFFICIENT STATISTICS
• X, f(x;),
• X1, X2,…,Xn
• Y=U(X1, X2,…,Xn ) is a statistic.
• A sufficient statistic, Y, is a statistic which contains all the information for the estimation of .
11
SUFFICIENT STATISTICS
• Given the value of Y, the sample contains no further information for the estimation of .
• Y is a sufficient statistic (ss) for if the conditional distribution h(x1,x2,…,xn|y) does not depend on for every given Y=y.
• A ss for is not unique:
• If Y is a ss for , then any 1-1 transformation of Y, say Y1=fn(Y) is also a ss for .
12
SUFFICIENT STATISTICS• The conditional distribution of sample rvs
given the value of y of Y, is defined as
1 21 2
; , , ,, , ,
;n
n
L x x xh x x x y
g y
1 21 2
, , , , ;, , ,
;n
n
f x x x yh x x x y
g y
• If Y is a ss for , then
1 21 2 1 2
; , , ,, , , , , ,
;n
n n
L x x xh x x x y H x x x
g y
ss for may include y or constant.
Not depend on for every given y.
• Also, the conditional range of Xi given y not depend on .
13
SUFFICIENT STATISTICS
EXAMPLE: X~Ber(p). For a r.s. of size n, show that is a ss for p.
n
1iiX
14
SUFFICIENT STATISTICS
• Neyman’s Factorization Theorem: Y is a ss for iff
1 2 1 2; , , , nL k y k x x x
where k1 and k2 are non-negative functions.
The likelihood function Does not depend on xi
except through y
Not depend on (also in the range of xi.)
15
EXAMPLES
1. X~Ber(p). For a r.s. of size n, find a ss for p if exists.
16
EXAMPLES
2. X~Beta(θ,2). For a r.s. of size n, find a ss for θ.
17
SUFFICIENT STATISTICS
• A ss may not exist.
• Jointly ss Y1,Y2,…,Yk may be needed. Example: Example 10.2.5 in Bain and Engelhardt (page 342 in 2nd edition), X(1) and X(n)
are jointly ss for
• If the MLE of exists and unique and if a ss for exists, then MLE is a function of a ss for .
18
EXAMPLE
X~N(,2). For a r.s. of size n, find jss for and 2.
MINIMAL SUFFICIENT STATISTICS
• If is a ss for θ, then,
is also a ss
for θ. But, the first one does a better job in data reduction. A minimal ss achieves the greatest possible reduction.
19
))x(s),...,x(s()x(S~
k~
1~
))x(s),...,x(s),x(s()x(S~
k~
1~
0~
*
20
MINIMAL SUFFICIENT STATISTICS• A ss T(X) is called minimal ss if, for any
other ss T’(X), T(x) is a function of T’(x).• THEOREM: Let f(x;) be the pmf or pdf of
a sample X1, X2,…,Xn. Suppose there exist a function T(x) such that, for two sample points x1,x2,…,xn and y1,y2,…,yn, the ratio
is constant with respect to iff T(x)=T(y). Then, T(X) is a minimal sufficient statistic for .
1 2
1 2
, , , ;
, , , ;n
n
f x x x
f y y y
21
EXAMPLE
• X~N(,2) where 2 is known. For a r.s. of size n, find minimal ss for .
Note: A minimal ss is also not unique. Any 1-to-1 function is also a minimal ss.
22
RAO-BLACKWELL THEOREM
• Let X1, X2,…,Xn have joint pdf or pmf f(x1,x2,…,xn;) and let S=(S1,S2,…,Sk) be a vector of jss for . If T is an UE of () and (T)=E(TS), then
i (T) is an UE of () .ii (T) is a fn of S, so it is also jss for .iii) Var((T) ) Var(T) for all . (T) is a uniformly better unbiased estimator
of () .
RAO-BLACKWELL THEOREM
• Notes:(T)=E(TS) is at least as good as T.
• For finding the best UE, it is enough to consider UEs that are functions of a ss, because all such estimators are at least as good as the rest of the UEs.
23
Example
• Hogg & Craig, Exercise 10.10
• X1,X2~Exp(θ)
• Find joint p.d.f. of ss Y1=X1+X2 for θ and Y2=X2.
• Show that Y2 is UE of θ with variance θ².
• Find φ(y1)=E(Y2|Y1) and variance of φ(Y1).
24
25
ANCILLARY STATISTIC
• A statistic S(X) whose distribution does not depend on the parameter is called an ancillary statistic.
• Unlike a ss, an ancillary statistic contains no information about .
Example
• Example 6.1.8 in Casella & Berger, page 257:
Let Xi~Unif(θ,θ+1) for i=1,2,…,n
Then, range R=X(n)-X(1) is an ancillary statistic because its pdf does not depend on θ.
26
27
COMPLETENESS• Let {f(x; ), } be a family of pdfs (or pmfs)
and U(x) be an arbitrary function of x not depending on . If
requires that the function itself equal to 0 for all possible values of x; then we say that this family is a complete family of pdfs (or pmfs).
0 for all E U X
0 for all 0 for all .E U X U x x
i.e., the only unbiased estimator of 0 is 0 itself.
28
EXAMPLES
1. Show that the family {Bin(n=2,); 0<<1} is complete.
29
EXAMPLES
2. X~Uniform(,). Show that the family {f(x;), >0} is not complete.
30
COMPLETE AND SUFFICIENT STATISTICS (css)
• Y is a complete and sufficient statistic (css) for if Y is a ss for and the family
; ;g y is complete. The pdf of Y.
1) Y is a ss for .
2) u(Y) is an arbitrary function of Y. E(u(Y))=0 for all implies that u(y)=0 for all possible Y=y.
31
BASU THEOREM• If T(X) is a complete and minimal sufficient
statistic, then T(X) is independent of every ancillary statistic.
• Example: X~N(,2).
: the mss for X
(n-1)S2/ 2 ~2
1n Ancillary statistic for
By Basu theorem, and S2 are independent.X
S2
statisticcompleteaisX
.familycompleteis)n/,(Noffamilyand)n/,(N~X 22
BASU THEOREM
• Example:
• Let T=X1+ X2 and U=X1 - X2
• We know that T is a complete minimal ss.
• U~N(0, 2) distribution free of T and U are independent by Basu’s Theorem
32
X1, X2~N(,2), independent, 2 known.
THE MINIMUM VARIANCE UNBIASED ESTIMATOR
• Rao-Blackwell Theorem: If T is an unbiased estimator of , and S is a ss for , then (T)=E(TS) is– an UE of , i.e.,E[(T)]=E[E(TS)]= and
– the MVUE of .
33
34
LEHMANN-SCHEFFE THEOREM
• Let Y be a css for . If there is a function Y which is an UE of , then the function is the unique Minimum Variance Unbiased Estimator (UMVUE) of .
• Y css for .
• T(y)=fn(y) and E[T(Y)]=.
T(Y) is the UMVUE of . So, it is the best estimator of .
35
THE MINIMUM VARIANCE UNBIASED ESTIMATOR
• Let Y be a css for . Since Y is complete, there could be only a unique function of Y which is an UE of .
• Let U1(Y) and U2(Y) be two function of Y. Since they are UE’s, E(U1(Y)U2(Y))=0 imply W(Y)=U1(Y)U2(Y)=0 for all possible values of Y. Therefore, U1(Y)=U2(Y) for all Y.
Example
• Let X1,X2,…,Xn ~Poi(μ). Find UMVUE of μ.
• Solution steps:– Show that is css for μ.
– Find a statistics (such as S*) that is UE of μ and a function of S.
– Then, S* is UMVUE of μ by Lehmann-Scheffe Thm.
36
n
1iiXS
Note
• The estimator found by Rao-Blackwell Thm may not be unique. But, the estimator found by Lehmann-Scheffe Thm is unique.
37
Recommended