SOLVED EXAMPLES

Rubber tires are known to have 20 000 km as mean life time and 5000 km as standard deviation. A Sample of 16 rubber tires are selected randomly, what is the probability that the SAMPLE MEAN will be in the interval from 17000 to 22000. Find the standard error in using the sample mean to ESTIMATE the population mean.

Example 1

937.00082.945.0)4.2()6.1()6.14.2(

)1250

2000022000

1250

2000017000()2000017000(

125016

5000

20000

)2200017000(

ZPS

XPXP

X

XP

n S

S deviation standard and mean with variablenormal is

Example 2

Suppose that X is the number of observed “successes” in a sample of n observations where p is the probability of success on each observation. a) Show that = X/n is an unbiased estimator of pb) Show that the standard error of is

p̂

npp /)1( p̂

npppnpn

XVarnnXVarpVarpSE

ppppEpBias

pnnpnXEpE

/)1()1(*)/1(

)()/1()/()ˆ()ˆ(

0)ˆ()ˆ(

//)()ˆ(

2

2

Let X be a random variable with mean and variance μ б2

Given two independent random samples of sizes n1 and n2, with sample means

Show that is an unbiased estimator of µ If are independent , Find the value of that minimizes the standard error of

10)1( 21 aXaXaX

21 XX ,

X

21 XX ,

0)()(

)1(

)()1()()( 21

XEXBias

aa

XEaXaEXE

2

22

1

22

22

12

)1(

)()1()()(

na

na

XVaraXVaraXVar

a

Example 3

21

1

22121

2

2

1

2

1)

11( 0

1

0)1(22)(

nn

na

nnna

n

a

n

a

na

naXVar

da

d

2

1

222

1

22

1

22

1

22

222

11

1

11

2

1

1

21

1

1

1

Bias

Xn

nX

nX

n

nX

n

nX

nS

XXXXn

XXn

S

SES

n

ii

n

ii

n

iii

n

ii

2

1

22

11

1XE

n

nXE

nSE

n

ii

Now Find 2iXE and 2

XE

5

2222

2222

ii

iiii

XXE

XEXEXEX

Var then,

Var

Therefore,

0

1

1

1111

11

1

222

22

22

22

222

2

1

22

SES

SE

n

nSE

n

n

nn

n

n

n

n

nSE

XEn

nXE

nSE

n

ii

Bias

22

2

222

nXE

nXEXEX

then,

Var

6

Then S2 is Unbiased Estimator of σ2

To proceed further to find SE(S2), Bias (S) and MSE(S),

a new important distribution should be considered.

It is the CHI-SQUARE distribution

Consider the variable ζ1 = Z2 where,

Z is Standard Normal variable

Z

ζ

0

σ2=1

1

2

n

Chi-square

Standard Normal

ζ1 is a Chi-square variable with one degree of freedom has mean=1 and variance = 2

22

212 ZZ

ζ2 is a Chi-square variable with two degrees of freedom has mean =2 and variance=4

223

22

21 .... nn ZZZZ

ζn is a Chi-square variable with n degrees of freedom has mean =n and variance = 2n

0

1

11

22

1

22

2

2

1

2

2

2

2

1

2

2

2

2

222

1

222

1

22

1

22

1

22

1

/

1

/1

1

1

1

11

1

21

1

1

1

1

1

1

1

n

ini

n

ii

n

i

i

n

i

i

i

n

iii

n

ii

n

ii

n

ii

ZZZSn

n

xxSn

n

x

n

x

n

S

xn

nx

nS

xxxxn

S

xxn

S

xxn

S

xxn

S

by sides both Devide

term, each Summing

is a Chi-square variable with (n-1) degrees of freedom ζn-1 .

Mean = n-1 and Variance = 2 (n-1)2

2)1(

Sn Then

Find the Standard Error SE of Estimator 2S

then,

22

4

2

4

12

42

1

22

12

2

1

2)(

1

2)1(2

11

1

)1(

nSVarSE

nn

nVar

nSVar

nS

Sn

n

n

n

9

BIAS and MEAN SQUARE ERROR OF ESTIMATOR S

0

1111

1

11

22

)(

1)(

11

nnnn

n

nn

dfE

En

SE

nS

nS

then

10

2

21

2

3

11

1

212

)(

n

en

f n

n

nn

The Probability density function of ζn-1 is given by:

Then

01

2

1)2/(

1

2)()(

nn

nSESBias

S is a Biased Estimator of σ

2

2

12

22)(

x

k

k

k ek

xxf

Chi-Square distribution

2

2

2

))(()()(

2

12/

1

21)(

,

SBiasSVarSMSE

nn

nSVar

Similarly

11

ESTIMATION OF NONCONVENTIONAL PARAMETERS

Conventional Parameters are mainly:Mean, Variance, Standard Deviation, Quartiles

Consider the following Population Probability Density Functions

0,10)(

0,101

)(

0,01

)(

1

/1

2

xxxf

xxxf

xexxfx

Parameters Ө,β and η in the above distribution are Nonconventional

There are TWO Methods for Deriving ESTIMATORSFor these PARAMETERS

1- Method of MOMENTS2- Method of MAXIMUM LIKELIHOOD FUNCTION

x

ex

xf1

)(

METHOD OF

MOMENTS

Consider the following Probability Density Function

0 ,0 1

)(2

xexxfx

Required to find an expression to Evaluate an Estimator of the Parameter θBy the method of Moments

n

ii

y

x

xn

x

xdyey

yxyx

put

dxexdxxfx

1

0

2

0

22

0

1

2

ˆ 2

1 2

1

)(

MAXIMUM LIKELIHOOD FUNCTION

Likelihood Function is the Joint Probability Function Of All points of the SAMPLE

Given a SAMPLE of size N: x1, x2, x3,…, xN

Taken from the population with known distribution f(x|Ө)

N

ii

N

xfL

xfxfxfxfL

1

321

)|()(

)|(.....*)|(*)|(*)|()(

The idea of the method of Maximum Likelihood is to determine

The expression of the parameter Ө that makes L(Ө) MAXIMUM

0)|()(1

N

iixfd

dL

d

d

This could be done by Differentiating L(Ө) w.r.t Ө and equating to zero

Example 1

A population is defined by the following probability density function

0,01

)(2

xexxfx

Find the Maximum Likelihood Estimator of Ө

N

ii

x

N

N

i

x

i xeexL

N

iii

1

1

21

21

11)(

Taking the logarithm of L(Ө)

N

ii

N

ii xLnxLnNLLn

11

)(ˆ1

)ˆ(2)ˆ(

Differentiate w.r.to Ө and equate to zero, we find

Consider the first moment of the f(x)

)ˆ()ˆ(

22ˆ,

0ˆ1

ˆ2

)ˆ(

1

12

EBias

X

N

xthen

xN

LLnd

d

N

ii

N

ii

NSE

NVar

XEEBias

2)ˆ(

4)ˆ(

02

)2()ˆ()ˆ(

2

22

1)(

22

3

22

0

then

then put

0

0

dyey

yxdxexdxxfx

y

x

Now, the Likelihood Function of the population is given by:

Therefore

Example 2A population is defined by the BINOMIAL distribution as follows:

NxppCxf xNxNx 01)( Find the Maximum Likelihood Estimator of p

The Likelihood Function of the population is given by:

M

j jj

x

MNM

M

j

xNx

jj

xNxp

ppNpL

ppxNx

NpL

M

jj

jj

1

1

)!(!

1ˆ1

ˆˆ1!)ˆ(

)ˆ1(ˆ)!(!

!)ˆ(

1

Taking the logarithm of the likelihood function

M

j jj

M

jj xNx

Lnp

pLnx

pLnNMNLnMpLLn

11 !!

1ˆ1

ˆ

ˆ1)!(ˆ

Differentiate w.r.to p̂

and equate to zero

0)ˆ()ˆ(

1)ˆ(

111ˆ

ˆ1

0ˆ1

ˆˆ1ˆ

ˆ1ˆ1

1

1

21

ppppEpN

NppE

NpN

XEN

pE

XN

xMN

p

pxNM

p

pp

p

px

p

NM

M

jj

M

jj

M

jj

Bias

then,

NpppSE

pNp

NXVar

NpVarpSE

/)1()ˆ(

)1(

1)(

1)ˆ()ˆ(

2/3

Example 3A population is defined by the following probability density function

10)1()( xxxf Find the Maximum Likelihood Estimator of Ө

N

ii

N

i

N

i

xxL1

ˆˆ

1

1ˆ)1ˆ()ˆ(

Now, the Likelihood Function of the population is given by:

Taking the logarithm of L(Ө)

i

i

xLnLnNLLn

xLnLnNLLn

ˆ1ˆ)ˆ(

1ˆ)ˆ(

Differentiate w.r.to Ө and equate to zero, we find

)(1

1ˆ1

0)(1ˆ

i

i

xLnN

xLnN

Therefore, the Maximum Likelihood Estimator MLE ̂

of the parameter Ө can be readily found as follows:

1)(

ˆ ixLn

N

A population is defined by the following Normal Distribution2

2

1

2

1)(

x

exf Find the Maximum Likelihood Estimators of µ, σ

Now, the Likelihood Function of the population is given by:

N

i

iix

NN

N

i

x

eeL 1

22

ˆ

ˆ

2

1

21

ˆ

ˆ

2

1

2ˆ

1

2ˆ

1)ˆ,ˆ(

Taking the logarithm of L(µ, σ)

N

i

ixLnN

NLnLLn1

2ˆ

2

1)2(

2)ˆ()ˆ,ˆ(

Differentiate partially w.r.to µ and σ and equate to zero, we find

N

ii

N

ii

N

i

i

xN

xx

LLn

1

11

2

1ˆ

0ˆˆ

2

1)ˆ,ˆ(

N

ii

i

xN

xNLLn

1

2

2

ˆ1

ˆ

0ˆ

ˆ

ˆ1

ˆ)ˆ,ˆ(

ˆ

Example 4