STATISTICS Sampling and Sampling Distributions Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University

STATISTICS Sampling and Sampling

Distributions

Professor Ke-Sheng ChengDepartment of Bioenvironmental Systems Engineering

National Taiwan University

Random sample • Let the random variables X1, X2, …, Xn have a

joint density that factors as follows:

where is the common density of each Xi . Then (X1, X2, …, Xn) is defined to be a random sample of size n from a population with density .

),,,(,,, 21 nXXXf

)()()(),,( 2121,,, 21 nnXXX xfxfxfxxxfn

)(f

)(f

04/10/23 2Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

• If X1, X2, …, Xn is a random sample of size n

from , then X1, X2, …, Xn are stochastically

independent. • Histogram -- A frequency (or relative

frequency) plot of observed data is called a frequency histogram (or relative frequency histogram).

)(f


Frequency Histogram


Cumulative frequency


Relative cumulative frequency


Statistic• A statistic is a function of observable random

variables, which is itself an observable random variable and does not contain any unknown parameters.

• A statistic must be observable because we intend to use it to make inferences about the density functions of the random variables.


• For example, if a random variable has a probability density function where and are unknown, then is not a statistic.

• If a statistic is not observable, then it can not be used to inference the parameters of the density function.

),( 2N

n

iiX

1

2


• An observation of random sample of size n can be regarded as n independent observations of a random variable.


• One of the central problems in statistics is to find suitable statistics to represent parameters of the probability distribution function of a random variable. Sample

Statistics

Population

Parameters

),( 2N

),( 2sx ),( 2

},,{ 1 nxx


Observable Unknown

Sample moments

• Let X1, X2, …, Xn be a random sample from the

density . Then the rth sample moment about 0 is defined as

)(f

n

i

rir X

nM

1

' 1


• In particular, if r = 1, we have the sample mean ; that is,

• Also, the rth sample moment about the sample mean is defined as

nX

n

iin X

nX

1

1

n

i

rnir XX

nM

1

)(1


• Theorem – Let X1, X2, …, Xn be a random

sample from the density . The expected value of the rth sample moment about 0 is equal to the rth population moment; i.e.,

Also,

)(f

'' ][ rrME

])([1

}])[(][{1

][

2''2

22

'

rrrr

r

nXEXE

n

MVar


• Special case: r=1

nnXVar

n

XEXEn

XVar

X /)(])([1

}])[(][{1

][

22'1

'2

22


Sample statistics

• Let X1, X2, …, Xn be a random sample from the

distribution of a random variable X. Sample mean and sample variance of the distribution are respectively defined to be

n

iiX

nX

1

1 2

1

2 )(1

1XX

nS

n

ii




Estimating the mean• Given a random sample from a probability

density function f( ． ) with unknown mean μ and finite variance σ2

, we want to estimate the mean using the random sample.

• Using only a finite number of values of X (a random sample of size n), can any reliable inferences be made about E(X), the average of an infinite number of values of X?

• Will the estimate be more reliable if the size of the random sample is larger?

nxxx ,, 21


R-program demonstration





Mean of sample means w.r.t. sample size

59.8

59.85

59.9

59.95

60

60.05

60.1

60.15

60.2

0 1000 2000 3000 4000 5000


Mean of sample standard deviations w.r.t. sample size

19.84

19.86

19.88

19.9

19.92

19.94

19.96

19.98

20

20.02

0 1000 2000 3000 4000 5000


Standard deviation of sample means w.r.t. sample size

y = 19.938x-0.4998

R2 = 0.9995

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0 1000 2000 3000 4000 5000

Y=f(x)=?What is the theoretical basis?


Histograms of sample mean and standard deviationns=30


Histograms of sample mean and standard deviationns=5000


Weak Law of Large Numbers (WLLN)

• Let f( ． ) be a density with mean μ and variance σ2, and let be the sample mean of a random sample of size n from f( ． ). Let ε and δ be any two specified numbers satisfying ε>0 and 0<δ<1. If n is any integer greater than , then

nX

2

2

1][ nXP


Recall the theorem



• (Example) Suppose that some distribution with an unknown mean has its variance equal to 1. How large a random sample must be taken such that the probability will be at least 0.95 that the sample mean will lie within 0.5 of the population mean?

nX

12 5.0

05.095.01

80)5.0)(05.0(

12

n


(Example) How large a random sample must be taken in order that you are 99% certain that is within 0.5σ of μ?nX

5.0

01.099.01 400

)5.0)(01.0( 2

2

n


Raingauge network design• Assuming there are already some raingauge

stations in a catchment, and we are interested in determining the optimal number of stations that should exist to achieve a desired accuracy in the estimation of mean rainfall.

• Two approaches– (1) Standard deviation of the sample mean should

not exceed a certain portion of the population mean.

– (2) 1][ nxP


Criterion 1Standard deviation of the sample mean should not exceed a

certain portion of the population mean.

2

22

,

),0(~)(,)/,(~

V

VX

nn

Cn

nC

n

nNXnNX

n


Criterion 2

• From the weak law of large numbers,

1][ nxP

2

2

n

What assumptions have we made for such approaches of network design ?

What are the practical considerations in monitoring network design?

Data independence


The Central Limit Theorem

• Let f( ． ) be a density with mean μ and finite variance σ2. Let be the sample mean of a random sample of size n from f( ． ). Then

approaches the standard normal distribution as n approaches infinity.

nX

n

XZ n

n


• The importance of the CLT is the fact that the mean of a random sample from any distribution with finite variance σ2

and mean μ is

approximately distributed as a normal random variable with mean μ and variance .

nX

n2

nN

nZX nn

,~


R-program demonstration- Central Limit Theorem





n=2n=10

n=25

n=50

n=100


Sampling distributions

• Given random samples of certain probability densities, we often are interested in knowing the probability densities of sampling statistics.– Poisson distribution– Exponential distribution– Normal distribution– Chi-square distribution– Standard normal and chi-square distributions– Student’s t-distribution


Poisson distribution






Exponential distribution


Normal distribution






Chi-square distribution



Standard normal and chi-square distributions



Student’s t-distribution

Student’s t distribution with k degrees of freedom


• The "student's" distribution was published in 1908 by W. S. Gosset. Gosset, however, was employed at a brewery that forbade the publication of research by its staff members. To circumvent this restriction, Gosset used the name "Student", and consequently the distribution was named "Student t-distribution.


Order statistics




Documents

STATISTICS Sampling and Sampling Distributions Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University