CHAPTER 10 ESTIMATION: THEORY

Gaza MATHEMATICAL STATISTICS –University Azhar-Al

Dr. Abuzaid. A. H. 29

CHAPTER 10

ESTIMATION: THEORY

10.1 Introduction:

Problems of statistical inference are divided into problems of estimation and tests of hypothesis.

The main difference between the two kinds is that in estimation we must define the value of a

parameter, whereas in tests of hypothesis we must decide whether to accept or reject a specific

value of a parameter.

Every random variable X , which represents the population, has p.d.f. with certain parameters.

These parameters are not always known. The problem of estimation is to estimate (determine) those

parameters.

The use of the value of a statistics to estimate a population parameter is known as point estimation.

The value of statistics is called a point estimate. For example, we use X to estimate the population

mean , or a value of 2S to estimate a population variance 2 .

The other type of estimation is known as "interval estimation", an interval estimate of is an

interval of the form 21

ˆˆ , where 1 and

2 are values of appropriate random variables.

Various statistical properties of estimators can be used to decide which estimator is most

appropriate in a given situation.

We may have more than one estimators of , the best estimator is that which give closed value to

the unknown parameter , with lowest cost.

10.2 Unbiased Estimators:

Definition: A statistic is an unbiased estimator of the parameter if and only if )ˆ(E .

Otherwise it called biased.

Example: If 521

X,...,X,X constitute a random sample from normal population with unknown

parameter . Check if the following estimator are unbiased or not?

1- 5

521

1

X...XXd

2- 3

21

22

XXX

d



Example: If X has a binomial distribution with the parameters n and , show that the sample

proportion n

X is an unbiased estimator of .

Solution:

Example: let

22

1 2xexp)x(f is a p.d.f., show that

n

ii

xn

d1

21 is an unbiased estimator

of .

Solution:

Example: Given a population of size N has the known mean and the finite variance 2 . Show

that

N

ii

)x(N 1

21 is an unbiased estimator of 2 .

Solution:



Theorem: If 2S is the variance of a random sample from an infinite population with the finite

variance 2 , then 2S is an unbiased estimator of 2 .

Proof:

Remark: in some cases for small sample size n, is an unbiased estimator of but )ˆ(E but

0

))ˆ(E(limn

. Then is called asymptotic unbiased estimator.

Example: If 521

X,...,X,X constitute a random sample from the population given by

elsewhere

x)x(f

0

01

Show that

12

1

n

xnd is a asymptotically unbiased of .

Solution:



10.3 Efficiency

We may have more than one unbiased estimators of , but which estimator is the best estimator?

the best estimator is the one whose sampling distribution has the smallest variance.

Example: let ),x(f is p.d.f for a random variable X , where )X(E , and )X(Var .

Three unbiased estimators of are given below:

3211xxxd

2312432 xxxd

3

321

3

xxxd

.

Show that the estimators of are unbiased, and Which is the most efficient estimator?

Solution:

Theorem: If is an unbiased estimator of and

2

1

)X(flnE..n

)ˆvar(

then is a minimum variance unbiased estimator of . Where n is the sample size.

Example: Show that X is a minimum variance unbiased estimator of the mean of a normal

population.



Example: Q 10.14: Show that n

X is a minimum variance unbiased estimator of the Bernoulli

parameter .

Solution:

If 1

and 2

are two unbiased estimators of the parameter of a given population and the

)ˆ(Var)ˆ(Var21

, then 1

is relatively more efficient than 2

. The ratio )ˆ(Var

)ˆ(Var

2

1

is a

measure of the efficiency of 2

relative to 1

.



Example. Q 10.25: If 21

X,X , and 3

X constitute a random sample of size n= 3 from a normal

population with the mean and the variance 2 , find the efficiency of 4

2321

XXX relative to

3

321XXX

.

Solution:

10.4 Consistency

Definition: The statistic is a consistent estimator of the parameter if and only if for each

0c

1

cˆPlimn

Theorem: If is an unbiased estimator of the parameter and 0

)ˆ(Varlimn

, then is a

consistent estimator of .

Example: Show that the sample variance 2S is a consistent estimator of 2 .

Solution:



Exercises:

Q.10.2 If 1

and 2

are two unbiased estimators of the same parameter , what condition must

be imposed on the constants 1

k and 2

k so that 2211

ˆkˆk is also an unbiased estimator of ?

Q. 10.6 Show that 2X is an asymptotically unbiased estimator of 2 .

Tutorial: 1,2,5,6,7,10,11,13-17, 23-25

METHODS OF FINDING THE ESTIMATOR OF

10.5 The method of moments

The method of moments is a classical way to get an estimator of a parameter which depends on

comparing the sample moments with the corresponding of the population.

The thr population moments are given by

x

rr'

r)x(fx)X(E if x is discrete.

dx)x(fx r if x is continuous.

110

0 )(E)X(E'

)X(E'

1 (population mean)

2

12

2 )( '' (population variance)



Definition: The thr sample moments of a set of observations n

x,...,x,x21

are given by

n

i

r

i

'

rx

nm

1

1

where r is the number of the unknown parameters. This means that, we have to write equations

according to the number of required parameter to estimate, and then we solve them.

Thus,

11

1

0

0

n

ii

'

n

nx

nm

xxn

mn

ii

' 1

1

1 (sample mean)

n

ii

' xn

m1

2

2

1

2

12

2 )m(ms '' (sample variance)

Example: For a given sample with size n, from the Bernoulli distribution with parameter . Use

the method of moments to obtain an estimator of .

Solution:

Example: For a given sample with size n, fro the gamma distribution with parameter and . Use

the method of moments for obtaining the estimates for and .

Solution:



Example: If n

x,...,x,x21

constitute a random sample of size n from a population with the mean

and the variance 2 , use the method of moments to find estimator for and 2 .

Solution:

Tutorial 10.53 – 10. 60

10.6 the method of maximum likelihood.

The maximum likelihood method which depends on finding the value of the unknown parameter

that maximize the joint distribution );x,....,x,x(fn

21.

Definition: If n

x,...,x,x21

are the values of the random sample from a population with the parameter

, the Likelihood function of the sample is given by

);x,....,x,x(f)(Ln

21 .

n

ii

)x(f as nXXX ,...,, 21 are independent random

variables



The maximum likelihood method for finding an estimator of , consist of finding the estimator

which make the function )(L is maximum. That is to find by finding 0)(ln

L.

Example: If n

x,...,x,x21

are the values of a random sample of the Poisson distribution

!x

exf

x

,....2,1,0x . Find the maximum likelihood estimator of .

Solution:

Example: If n

x,...,x,x21

are the values of a random sample of size n, from the exponential

population

x

exf

1

, 0x . Find the maximum likelihood estimator of .

Solution:



Example: If n

x,...,x,x21

are the values of a random sample of size n, from the Bernoulli population

xx )(xf 11 , 10,x . Find the maximum likelihood estimator of .

Solution:



Example: If n

x,...,x,x21

are the values of a random sample of size n, from the normal population.

Find the maximum likelihood estimator of :

1- if 2 is known.

2- 2 if is known.

3- If and 2 are known.

Solution:



CHAPTER 11

ESTIMATION: APPLICATIONS

11.1 Introduction:

The point estimate is a single number, thus, it does not provide any information about the

precision and reliability of estimation. Alternatively, we might use interval estimation.

An interval estimate of is an interval of the form 21

ˆˆ , where 1 and

2 are values

of appropriate random variables, where 1)ˆˆ(21

P .

We refer to 21ˆˆ as a %100)1( confidence interval for .

)1( is called the degree of confidence, and the endpoint of the interval, 1 and 2 , are

called the lower and upper confidence limits respectively.

When 05.0 , the degree of confidence is 0.95 and we get 95% confidence interval.

Interval estimates of a given parameter are not unique.

11.2 Estimation of Means:

The sampling distribution of X for a random samples of size n from a normal population with the

mean and the variance 2 is a normal distribution with x and n

x

22

. Thus, we can

write

12zZP

where, n

XZ

/

, and 2z is the area under the standard normal curve from 2z to . It follows

that

1.

2

nzXP

then the maximum error will be at most,n

z

.2 .

The values of z for the 95% level of confidence



Example: For a random sample of size n =150, derived from a normal distribution with standard

deviation, = 6.2, what can you assert with probability 0.99 about the maximum error of the

estimate.

Solution:

Example: In the previous example, find the maximum error of the estimate with

1. probability 0.95.

2. the sample size n =100.

3. the standard deviation, = 8.

Solution:

The factors that determine the width of a confidence interval are:

1.The sample size, n.

2.The variability in the population, usually σ estimated by s.

3.The desired level of confidence.

11.2.1. The confidence Interval for the Mean of a normal population with known variance

Theorem: If x is the value of the mean of a random sample of size n from a normal population

with the known variance 2 , then a %100)1( confidence interval for the mean of the population

is given by

nzx

nzx

.. 22

Proof:



Example: If a random sample of size n = 20 from a normal population with the variance 2 = 225

has the mean x = 64.3, construct a 95% confidence interval for the population mean .

Solution:

Example: Q.11.10 A district official intends to use the mean of a random sample of 150 from a very

large school district to estimate the mean score. If, based on experience, it is known that σ = 9.4 for

such data, what can she assert with probability 0.95 about the maximum error.

Solution:

The size of a random sample derived from a normal distribution with standard deviation σ,

%100)1( confidence level and maximum error , is given by 2

2

zn

Example: what is the sample size which can be used for 95% confidence interval when differ

from x by less than 0.06 and σ = 0.3.

Solution:

11.2.2 The confidence Interval for the Mean of a normal population with unknown variance

Theorem: For a sample of size )30( n from a normal population with unknown variance 2 ,

then it is reasonable to estimate the population variance 2 by the sample variance 2s , then a

%100)1( confidence interval for the mean of the population is given by

n

szx

n

szx .. 22

Theorem: For a sample of size )30( n from a normal population with unknown variance 2 ,

then it is reasonable to estimate the population variance 2 by the sample variance 2s , then a

%100)1( confidence interval for the mean of the population is given by

n

stx

n

stx nn .. 1,21,2



The values of t for the 95% level of confidence

Example: You work for an airline and are studying the time it takes for passengers to get their

baggage after leaving a plane. You know that σ = 6.4 min. You take a random sample of 45

measurements and get a time of x = 26.7 min. What is the approximate 99% confidence interval

for the population mean time ?

Solution:

Example: A random sample of size n = 16 is taken from a normally distributed population with

unknown and σ. If the sample has a mean x = 27.9 and a standard deviation s = 3.23, then what

is the 99% confidence interval for the data?

Solution:

Theorem: %100)1( confidence interval for when σ is known and the sample is drawn

without replacement ( from finite population with size N) is given by

1.

1. 22

N

nN

nzx

N

nN

nzx

If 2 is unknown and )30( n , then a %100)1( confidence interval for is given by

1.

1. 22

N

nN

n

szx

N

nN

n

szx

If 2 is unknown and )30( n , then a %100)1( confidence interval for is given by



1.

1. 1,21,2

N

nN

n

stx

N

nN

n

stx nn

The maximum error of the estimation with probability )1( is given by 1

.2

N

nN

nz

Example: Q.11.17 A district official intends to use the mean of a random sample of 150 from a very

large school district with size N = 900 to estimate the mean score and she gets x = 61.8. If, based

on experience, it is known that σ = 9.4 for such data. Construct 99% confidence interval for .

Solution:

11.3 The Estimation of Differences Between Means:

For independent random samples from normal populations

2

2

2

1

2

1

2121)()(

nn

XXz

has the standard normal distribution.

Case 1: With known variances

Theorem: If 1

x and 2

x are the values of the means of independent random samples of size 1

n and

2n from normal populations with the known variances 2

1 and 2

2 , then a %100)1( confidence

interval for the difference between the two population means is given by

2

2

2

1

2

1

2/2121

2

2

2

1

2

1

2/21)()(

nnzxx

nnzxx

The maximum error of the difference between two means with probability )1( is given by

2

2

2

1

2

1

2/nn

z

Example, Q. 11.23: Independent random samples of size 1

n =16 and 2

n =25 from normal

populations with 1

= 4.8 and 2

=3.5 have the means 1

x =18.2 and 2

x =23.4.

1- What can you assert with 98% confidence interval about the maximum error.

2- Find a 90% confidence interval 21 .



Case 2: With unknown variances and large sample size

Theorem: If 1

x and 2

x are the values of the means of independent random samples of size

)30(1n and )30(

2n from normal populations with the unknown variances 2

1 and 2

2 , then a

%100)1( confidence interval for the difference between the two population means is given by

2

2

2

1

2

1

2/2121

2

2

2

1

2

1

2/21)()(

n

s

n

szxx

n

s

n

szxx

The maximum error of the difference between two means with probability )1( is given by

2

2

2

1

2

1

2/n

s

n

sz

Case 3: With unknown and unequal variances 2

2

2

1

Theorem: If 1

x and 2

x are the values of the means of independent random samples of size 1

n and

2n from normal populations with the unknown and unequal variances and at least one of the two

samples has size less than 30, then a %100)1( confidence interval for the difference between the

two population means is given by

2

2

2

1

2

1

,2/2121

2

2

2

1

2

1

,2/21)()(

n

s

n

stxx

n

s

n

stxx

where

11

)(

2

2

2

1

2

1

2

21

n

w

n

w

ww and

1

2

1

1n

sw ,

2

2

2

2n

sw .



Case 4: With unknown and equal variances 22

2

2

1

Theorem: If 1

x and 2

x are the values of the means of independent random samples of sizes 1

n and

2n from normal populations with the unknown and equal variances and at least one of the two

samples has size less than 30, then, then a %100)1( confidence interval for the difference

between the two population means is given by

21

)2,2/(2121

21

)2,2/(21

11.)(

11.)(

2121 nnstxx

nnstxx

pnnpnn

where p

s is the pooled standard deviation which is an estimate of the variance , and can be

obtained by 2

)1()1(

21

2

22

2

11

nn

snsns

p.

Example: An experiment was conduced in which two types of engines, A and B were compared. 75

experiments were conducted using engine type A and 50 experiments were done for engine type B.

The average gas mileage for engine A was 42 miles per gallon and for engine B was 36 miles per

gallon. The sample standard deviations are 8 and 6 for engine A and B respectively. Assuming that

the two populations sampled are normal and have unequal variance, find a 90% confidence interval

for the difference between the average gas mileages of the two types of engine.

Solution:

Example: 12 randomly selected mature citrus trees of one variety have a mean height of 13.8 feet

with a standard deviation of 1.2 feet, and 15 randomly selected mature citrus trees of another variety

have a mean height of 12.9 feet with a standard deviation of 1.5 feet. Assuming that the random

samples were selected from normal populations with equal variances. Construct a 95% confidence

interval for the difference between the true average heights of the two kinds of citrus trees.

Solution:



Example: 12 randomly selected mature citrus trees of one variety have a mean height of 13.8 feet

with a standard deviation of 1.2 feet, and 15 randomly selected mature citrus trees of another variety

have a mean height of 12.9 feet with a standard deviation of 1.5 feet. Assuming that the random

samples were selected from normal populations with unequal variances. Construct a 95%

confidence interval for the difference between the true average heights of the two kinds of citrus

trees.

Solution:

Tutorial: 11.10 11.27



11.4 The Estimation of The Proportions

A point estimator of the population in a binomial experiment is given by the statistic n

X

where X represents the number of successes in n trials.

If 5ˆ n and 5)ˆ1( n , by central limit theorem, for n sufficiently large, is approximately

normally distributed with mean;

)ˆ(ˆ E and variance n

)1()ˆvar(2

ˆ

. Therefore,

).1,0(~)1(

ˆ

)ˆvar(

)ˆ(ˆN

n

EZ

Theorem: If is the proportion of successes in a random sample of size n, then an approximate

%100)1( confidence interval for the binomial parameter is given by

nz

nz

)ˆ1(ˆ.ˆ)ˆ1(ˆ

.ˆ2/2/

Example: A DNA test on n = 48 trials samples at a laboratory reports 16 with trace element in the

suspect’s DNA. Let = 16/48=0.333 be the point estimate of the success .

Construct a confidence interval for with the confidence level of approximately 95%.

Solution:

Example: In a random sample of size 500 persons. It is found that 160 persons have a computers.

Construct 95% confidence interval for the true proportion of persons have computers.

Solution:

11.5 The Estimation of Differences Between Proportions:

Theorem: If 1 and

2 are the proportion of successes in a random sample of size

1n and

2n , then

an %100)1( confidence interval for the binomial parameter 21

is given by

2

22

1

11

2/2121

2

22

1

11

2/21

)ˆ1(ˆ)ˆ1(ˆ.)ˆˆ(

)ˆ1(ˆ)ˆ1(ˆ.)ˆˆ(

nnz

nnz



Example: In a random sample of 400 adults and 600 teenagers who watched a certain TV program,

100 adults and 300 teenagers indicated that they liked it. Construct

(a) 95%,

(b) 99%

confidence limits for the difference in proportions of all adults and all teenagers who watched the

program and liked it.

Solution:

Tutorial: Q. 11.34 Q.11.40 and Q. 11.45 Q.11.48

11.6 The Estimation of Variances Given a random sample of size n from a normal population, we can obtain a %100)1(

confidence interval for 2 by making the use of the theorem that 2

12

2

~)1(

n

Sn.

Theorem: If 2s is the value of the variance of a random sample of size n from a normal population,

then

2

1,2/1

2

2

2

1,2/

2 )1()1(

nn

snsn

is a %100)1( confidence interval for 2 .

Proof



Example: A random sample of size n =16, was taken from a normal population with mean and

variance 2 . If the sample standard deviation s = 2.2. Construct 99% confidence interval for 2 .

Solution:

11.7 The Estimation of The Ratio of Two Variances

Theorem: If 2

1s and 2

2s are the values of the variances of independent random samples of size

1n

and 2

n from normal population, then

1,1,2/2

2

2

1

2

2

2

1

1,1,2/

2

2

2

1

12

21

.1

.

nn

nn

fs

s

fs

s

is a %100)1( confidence interval for 2

2

2

1

.

Note: 1,1,2/ 21

1

nnf

= 1,1,2/1 12 nn

f

Proof:



Example: a random sample of size 9 is drawn from normal population if the observed values of this

random is 16.4,15.8, 17, 15.9, 15.8, 16.9, 15.2, 16, 16.1 another random sample of size 10, was

taken from another population gives sample variance 0.25. construct 90% confidence interval for

2

2

2

1

.

Solution:

Tutorial: Q. 11.51 Q.11.57

Documents

CHAPTER 10 ESTIMATION: THEORY