Embed Size (px)
Citation preview
1
Sampling and Sampling Distributions
Dr. Jerrell T. Stracener, SAE Fellow
Leadership in Engineering
EMIS 7370/5370 STAT 5340 : PROBABILITY AND STATISTICS FOR SCIENTISTS AND ENGINEERS
Systems Engineering ProgramDepartment of Engineering Management, Information and Systems
2
Populationthe total of all possible values (measurement, counts, etc.) of a particular characteristic for aspecific group of objects.
Samplea part of a population selected according to some rule or plan.
Why sample?- Population does not exist- Sampling and testing is destructive
Population vs. Sample
3
Characteristics that distinguish one type of sample from another:
• the manner in which the sample was obtained
• the purpose for which the sample was obtained
Sampling
4
• Simple Random SampleThe sample X1, X2, ... ,Xn is a random sample if X1, X2, ... , Xn are independent and identically distributed random variables.
Remark: Each value in the population has an equal and independent chance of being included in the sample.
•Stratified Random SampleThe population is first subdivided into sub-populations for strata, and a simple randomsample is drawn from each strata
Types of Samples
5
Censored Samples• Type I Censoring - Sample is terminated at a fixed time, t0. The sample consists of K times to failure plus the information that n-k items survived the fixed time of truncation.
• Type II Censoring - Sampling is terminated upon the Kth failure. The sample consists of K times to failure, plus information that n-k items survived the random time of truncation, tk.
• Progressive Censoring - Sampling is reduced in stage.
Types of Samples (continued)
6
• Systematic Random Sample
The N items in the population are arranged in some order.
Select an item at random from the first K = N/n items, where n is the sample size.
Select every Kth item thereafter.
Types of Samples (continued)
7
Sampling -Monte Carlo Simulation
8
For any random variable Y with probability densityfunction f(y), the variable
is uniformly distributed over (0, 1), or F(y) has theprobability density function
y
dxxfyF )()(
1y0for 1)( yFg
Uniform Probability Integral Transformation
9
Remark: the cumulative probability distributionfunction for any continuous random variable isuniformly distributed over the interval (0, 1).
Uniform Probability Integral Transformation
10
f(y)
F(y)y
y
1.00.80.60.40.2 0
ri
yi
Generating Random Numbers
11
Generating values of a random variable using theprobability integral transformation to generate arandom value y from a given probability densityfunction f(y):
1. Generate a random value rU from a uniformdistribution over (0, 1).
2. Set rU = F(y)
3. Solve the resulting expression for y.
Generating Random Numbers
12
From the Tools menu, look for Data Analysis.
Generating Random Numbers with Excel
13
If it is not there, you must install it.
Generating Random Numbers with Excel
14
Once you select Data Analysis, the following window will appear. Scroll down to “Random Number Generation” and select it, then press “OK”
Generating Random Numbers with Excel
15
Choose which distribution you would like. Use uniform for an exponential or weibull distribution or normal for a normal or lognormal distribution
Generating Random Numbers with Excel
16
Uniform Distribution, U(0, 1). Select “Uniform” under the “Distribution” menu.Type in “1” for number of variables and 10 for number of random numbers. Then press OK. 10 random numbers of uniform distribution will now appear on a new chart.
Generating Random Numbers with Excel
17
Normal Distribution, N(μ, σ). Select “Normal” under the “Distribution” menu.Type in “1” for number of variables and 10 for number of random numbers. Enter the values for the mean (m) and standard deviation (s) then press OK. 10 random numbers of uniform distribution will now appear on a new chart.
Generating Random Numbers with Excel
18
First generate n random variables, r1, r2, …, rn, from
U(0, 1). Select “Uniform” under the “Distribution” menu.Type in “1” for number of variables and 10 for number of random numbers. Then press OK. 10 random numbers of uniform distribution will now appear on a new chart.
Generating Random Values from an ExponentialDistribution E() with Excel
19
Select a θ that you would like to use, we will use θ = 5.
Type in the equation xi= -ln(1 - ri), with filling in θ as 5, and ri as cell A1 (=-5*LN(1-A1)). Now with that cell selected, place the cursor over the bottom right hand corner of the cell. A cross will appear, drag this cross down to B10. This will transfer that equation to the cells below. Now we have n random values from the exponential distribution with parameter θ=5 in cells B1 - B10.
Generating Random Values from an ExponentialDistribution E() with Excel
20
First generate n random variables, r1, r2, …, rn, from U(0, 1).
Select “Uniform” under the “Distribution” menu.
Type in “1” for number of variables and 10 for number of random numbers. Then press OK. 10 random numbers of uniform distribution will now appear on a new chart.
Generating Random Values from an WeibullDistribution W(β, ) with Excel
21
Select a β and θ that you would like to use, we will use β =20, θ = 100.
Type in the equation xi = [-ln(1 - ri)]1/, with filling in β as 20, θ as 100, and ri as cell A1 (=100*(-LN(1-A1))^(1/20)). Now transfer that equation to the cells below. Now we have n random variables from the Weibull distribution with parameters β =20 and θ =100 in cells B1 - B10.
Generating Random Values from an WeibullDistribution W(β, ) with Excel
22
First generate n random variables, r1, r2, …, rn, from N(0, 1).
Select “Normal” under the “Distribution” menu.
Type in “1” for number of variables and 10 for number of random numbers. Enter 0 for the mean and 1 for standard deviation then press OK. 10 random numbers of uniform distribution will now appear on a new chart.
Generating Random Values from an LognormalDistribution LN(μ, σ) with Excel
23
Select a μ and s that you would like to use, we will use μ = 2, σ = 1.
Type in the equation , with filling in μ as 2, σ as 1, and ri as cell A1 (=EXP(2+A1*1)). Now transfer that equation to the cells below. Now we have an Lognormal distribution in cells B1 - B10.
iri ex
Generating Random Values from an LognormalDistribution LN(μ, σ) with Excel
24
Flow Chart of Monte Carlo Simulation method
Input 1: Statistical distribution for each component variable.
Input 2: Relationshipbetween componentvariables and systemperformance
Select a random value from each of these distributions
Calculate the value of system performance for a system composed of components with the values obtained in the previous step.
Output: Summarize and plot resultingvalues of system performance. Thisprovides an approximation of the distribution of system performance.
Repeatntimes
25
Because Monte Carlo simulation involves randomlyselected values, the results are subject to statisticalfluctuations.
• Any estimate will not be exact but will have anassociated error band.
• The larger the number of trials in the simulation, the more precise the final results.
• We can obtain as small an error as is desired byconducting sufficient trials
• In practice, the allowable error is generally specified,and this information is used to determine the required trials
Sample and Size Error Bands
26
Example
24696.105.0
8.02.0 2
2 n
If X~ B(n,p) and the desired confidence level is 95%,then 1 - = 0.95 and = 0.05 and Z1-/2 = 1.96; and if = 0.2. Then an estimate of the required sample size is
'P
27
• there is frequently no way of determining whether any of the variables are dominant or more important than others without making repeated simulations
• if a change is made in one variable, the entiresimulation must be redone
• the method may require developing acomplex computer program
• if a large number of trials are required, a great deal of computer time may be needed to obtainthe necessary results
Drawbacks of the Monte Carlo Simulation
28
If the probability density function of X is
Find
(a) F(x)
(b) Mean
(c) Standard Deviation
(d) The value of x for which P(X > x)=0.05
(e) If 5 values of x are randomly selected find the
probability that at least 2 of them will exceed 0.6
(f) Redo parts (a) thru (e) using Monte Carlo Simulation
)(xf)1(2 x
0
1x0for elsewhere
Example
29
First, plot : )(xf
0
0.5
1
1.5
2
0 0.2 0.4 0.6 0.8 1
x
f(x
)
Example - Solution
30
(a) The (cumulative) probability distribution function of X for is 10 x
)(F x
2
0
2
0
0 0
0
2
222
)(22
)1(2
)(
)X(P
xx
yy
dyydy
dxy
dxxf
x
x
x
x x
x
x
Example - Solution
31
so that
)(xF 22 xx 1
1x0for 1for x
0 0for x
Example - Solution
32
(b) The mean of X is
)(XE
3
1
3
1
2
12
322
][2
)1(2
1
0
32
1
0
2
1
0
xx
dxxx
dxxx
Example - Solution
33
222 )(E)( XXVar
18
1
9
1
12
2
3
1
4
1
3
12
9
1
432
9
1)1(2
3
1)(
2
1
0
43
1
0
2
21
0
2
xx
dxxx
dxxfx
The variance of X is
Example - Solution
34
The standard deviation is
)(VAR X
236.018
118
1
Example - Solution
35
(d) The value of x such that P(X > x) = 0.05 can be determined by a couple of different approaches.
x can be obtained by solving the following equation for x,
or by solving F(x) = 0.95 for x,
)( xXP 1
05.0)(x
dyyf
)(XF 2295.0 xx
Example - Solution
36
Here
its roots are
1.2236 is outside of our range, so is our answer. If we check with our plot of the data, this seems reasonable.
7764.0or 2236.1 xx
095.022 xx
7764.0x
0.7764
Example - Solution
00.20.40.60.8
11.21.41.61.8
2
0 0.2 0.4 0.6 0.8 1
x
f(x)
0.05
37
(e) Let Y = number of values that exceed 0.6, for y = 0,1,2,3,4,5.
Now
)6.0( XP
16.0
84.01
6.0)6.0(21
)6.0(1
)6.0(1
2
F
XP
16.0,5B~Y
Example - Solution
38
so that
)2( YP
1835.0
3983.04182.01
16.0116.05
1
16.0,5;1
51
0
1
0
yy
y
y
y
xb
Example - Solution
39
(f) Generate a random sample of n, say 1,000, from using Monte Carlo Simulation as follows:
Since
generate
and solve for xi
)(xf
1,0for 2)( 2 xxxxF
0,1 Ufrom ir
1,...,1000for 2 2 irxx iii
Example - Solution
40
Then estimate F(x), μ, σ and as follows:
10for 1000
)(ˆ10
1
xf
xFk
i
2YP
Interval Frequency, f i
0-0.1 196 0.196 0.1960.1-0.2 170 0.17 0.3660.2-0.3 136 0.136 0.5020.3-0.4 119 0.119 0.6210.4-0.5 103 0.103 0.7240.5-0.6 96 0.096 0.820.6-0.7 78 0.078 0.8980.7-0.8 56 0.056 0.9540.8-0.9 35 0.035 0.9890.9-1.0 11 0.011 1
0.0-1.0 1000 1
1000if
1 0 0 0if
0
0.05
0.1
0.15
0.2
0-0.1
0.1-0.2
0.2-0.3
0.3-0.4
0.4-0.5
0.5-0.6
0.6-0.7
0.7-0.8
0.8-0.9
0.9-1.0
x
rela
tive f
req
uen
cy
f(x)
Example - Solution
41
10for 1000
)(ˆ10
1
xf
xFk
i
Then estimate F(x), μ, σ and as follows: 2YP
0
0.2
0.4
0.6
0.8
1
0-0.1
0.1-0.2
0.2-0.3
0.3-0.4
0.4-0.5
0.5-0.6
0.6-0.7
0.7-0.8
0.8-0.9
0.9-1.0
x
)(ˆ xF
F(x)
Example - Solution
42
Compare this to =
μ̂
34079.0
79.3401000
1
1000
1 1000
11
i
x
3
1
Example - Solution
43
where
σ̂n
nS
1
2S
0599.0
9991000
34.11613993.1751000
1
22
nn
xxn ii
S
2446.0
0599.0
2
S
Example - Solution
44
Compare this to = 0.236 .
σ̂
2445.01000
9992446.0
1
n
nS
Example - Solution
45
Compare this to the
p̂
18.01000
180 valuesofnumber total
0.6 of valuesof no.
0.6XP̂
x
0.16p
Example - Solution
46
Compare this to the 1835.0P
)2(ˆ YP
20.0200
001192
groups ofnumber total
0.6 of values2 have that 5 of groups theno.
x
Example - Solution
47
Remember, that there are 1000 points of data that we have used. To access our data, just double click on the excel chart to the left.
ri xi >0.6
num in group >0.6
0.38200 0.21387 00.10068 0.05168 00.59648 0.36477 00.89911 0.68236 10.88461 0.66031 1 20.95846 0.79620 10.01450 0.00727 00.40742 0.23021 00.86325 0.63020 10.13858 0.07188 0 20.24503 0.13111 00.04547 0.02300 00.03238 0.01632 00.16413 0.08574 00.21961 0.11660 0 00.01709 0.00858 00.28504 0.15445 00.34309 0.18950 00.55364 0.33190 00.35737 0.19836 0 00.37184 0.20743 00.35560 0.19726 00.91031 0.70051 10.46602 0.26926 00.42616 0.24248 0 10.30390 0.16568 00.97571 0.84414 10.80667 0.56030 0
0
0. 2
0. 4
0. 6
0. 8
1
1 . 2
1
Example - Solution - Our Data
48
Sampling Distributions
49
If X1, X2, ... ,Xn is a random sample of size n from a normal distribution with mean andknown standard deviation ,
and if
then
n
σμ,N~X
0,1N~
n
σμX
Z
and
n
1iiX
n
1X
Sampling Distribution of with known X
50
The dollar amount per transaction, X, in the Sporting Goods Department of a store has a normaldistribution with mean $75 and standard deviationof $20. What is the probability that a random sample of 9 sales transactions will have an average over $85?
Sampling Distributions: Example
51
If X ~ N(75, 20), then
9
2075,N~X
9
207585
n
σμX
P85XP
1.5ZP
0668.0
Sampling Distributions: Example - Solution
52
If is the mean of a random sample of size n, X1, X2, …, Xn, from a population with mean and finite standard deviation , then if n the limiting distribution of
n
XZ
is the standard normal distribution.
Central Limit Theorem
X
53
Remark: The Central Limit Theorem provides the basis for approximating the distribution of X witha normal distribution with mean and standard deviation
The approximation gets better as n gets larger.
n
Central Limit Theorem
54
A manufacturing process produces parts with a mean diameter of 5 mm. An engineer conjectures that the population mean is 5.0 mm, and an experiment is conducted in which 100 parts are selected randomly and measured. It is known that the population = 0.1. The experiment indicates a sample average diameter = 5.027 mm. Does this refute the engineer’s conjecture?
Solution: Whether or not the data support or refute the conjecture depends on the probability that data similar to that obtained in this experiment can readily occur when = 5.0. In other words, how likely is it that one can obtain 5.027 with n = 100 if the mean is equal to = 5.0?
Central Limit Theorem - Example
X
X
55
The probability that we choose to compute is given by P[( - 5) 0.027]. This is the same as asking, if the mean is 5, what is the chance that it will deviate by so much as 0.027?
]027.0)5[(XP ]027.0)5[( XP
7.2
100/1.0
5]027.0)5[(
XPXP
Solution
X
56
Here we are simply standardizing the sample mean according to the Central Limit Theorem.
]7.2[7.2100/1.0
5
ZP
XP
0035.0
Thus one would experience by chance a sample mean that is 0.027 mm from the population mean in only about 3.5 of 1000 experiments. Therefore the sample data does not support the engineer’s conjecture.
Solution (Continued)
57
Let X1, X2, ..., Xn be independent random variables that have normal distribution with mean and unknown standard deviation . Let
and
n
1iiX
n
1X
Then the random variable
n
1i
2
i2 XX
1n
1S
n
SμX
T
has a t-distribution with = n - 1 degrees of freedom.
Sampling Distribution of with Unknown X
58
If S2 is the variance of a random sample of size n taken from a normal population having variance 2, then the statistic
n
i
i XX
12
2
2
22 s 1n
has a chi-squared distribution with = n - 1 degrees of freedom.
Sampling Distributions of S2
59
A manufacturer of car batteries guarantees that his product will last, on average, 3 years with a standard deviation of 1 year. If five batteries have lifetimes of 1.9, 2.4, 3.0, 3.5 and 4.2 years, is the manufacturer still convinced that his batteries have a standard deviation of 1 year? Assume that battery lifetime follows normal distribution .
Solution: We first find the sample variance:
815.0
45
1526.485 22
s
Example
60
Then
26.3
1
815.042
is a value from a chi-squared distribution with 4 degrees of freedom. Since 95% of the 2 values with 4 degrees of freedom fall between 0.484 and 11.143, the computed value with 2 = 1 is reasonable, and therefore the manufacturer has no reason to suspect that the standard deviation is other than 1 year.
Solution (Continued)
