Introduction to Probability and Statistics Thirteenth Edition Chapter 11 The Analysis of Variance

Introduction to Probability Introduction to Probability and Statisticsand Statistics

Thirteenth EditionThirteenth Edition

Chapter 11

The Analysis of Variance

Experimental DesignExperimental Design

• The sampling plansampling plan or experimental designexperimental design determines the way that a sample is selected.

• In an observational studyobservational study,, the experimenter observes data that already exist. The sampling plansampling plan is a plan for collecting this data.

• In a designed experimentdesigned experiment,, the experimenter imposes one or more experimental conditions on the experimental units and records the response.

• Must design an experiment that will test your hypothesis.

• This experiment will allow you to change some conditions or variables to test your hypothesis.

Experimental DesignExperimental Design

The independent variable is the variable that is purposely changed. It is the manipulated variable.

The dependent variable changes in response to the independent variable. It is the responding variable.

VariablesVariables are things that change.

The Basic Principles of Experimental Design

• Two aspects to any experimental problem:1. The design of the experiment2. Statistical analysis of the data

• Three basic principles of experimental design1. Replication2. Randomization3. Blocking

To reduce error

Definitions

• An experimental unitexperimental unit is the object on which a measurement or measurements) is taken.

• A factorfactor is an independent variable whose values are controlled and varied by the experimenter.

• A levellevel is the intensity setting of a factor.• A treatmenttreatment is a specific combination of

factor levels. • The responseresponse is the variable being

measured by the experimenter.

Example

• A group of people is randomly divided into an experimental and a control group. The control group is given an aptitude test after having eaten a full breakfast. The experimental group is given the same test without having eaten any breakfast.

Experimental unit : Factor =

Response : Levels =

Treatments :

person

Score on test

meal

Breakfast or no breakfast

Breakfast or no breakfast

• The experimenter in the previous example also records the person’s gender. Describe the factors, levels and treatments.

Experimental unit = Response =

Factor #1 = Factor #2 =

Levels = Levels =

Treatments:

person score

meal

breakfast or no breakfast

gender

male or female

male and breakfast, female and breakfast, male and no breakfast, female and no breakfast

Example

The Analysis of Variance (ANOVA)

All measurements exhibit variabilityvariability..The total variation in the response

measurements is broken into portions that can be attributed to various factorsfactors.

These portions are used to judge the effect of the various factors on the experimental response.

The Analysis of Variance

If an experiment has been properly designed,

Total variationTotal variationFactor 2Factor 2

Random variationRandom variation

Factor 1Factor 1

•We compare the variation due to any one factor to the typical random variation in the experiment.The variation between the sample means is

larger than the typical variation within the samples.

The variation between the sample means is about the same as the typical variation within the samples.

Factor …Factor …

Assumptions

1. The observations within each population are normally distributed with a common variance 2.

2. Assumptions regarding the sampling procedures are specified for each design.

1. The observations within each population are normally distributed with a common variance 2.

2. Assumptions regarding the sampling procedures are specified for each design.

Analysis of variance procedures are fairly robust when sample sizes are equal and when the data are fairly mound-shaped.

Two Designs

1.1. Completely Randomized Design Completely Randomized Design (CRD)(CRD)an extension of the two independent sample t-test.

2.2. Completely Randomized Block Completely Randomized Block Design (CRBD) Design (CRBD) an extension of the paired difference test.

1. The Completely Randomized Design 1. The Completely Randomized Design (CRD)(CRD)

• A one-way classificationone-way classification in which one factor is set at k different levels.

• The k levels correspond to k different normal populations, which are the treatmentstreatments.

• Are the k population means the same, or is at least one mean different from the others?

Randomization in CRDRandomization in CRD

• Factor : A

• A has 4 levels (a1, a2,a3, and a4)

• There are 4 replications each level (balance)

• So, there fill be 4 x 4 = 16 EUa4

a4

a4 a4

a2

a2

a2

a2

a3

a3 a3

a3

a1

a1

a1 a1

AssumptionAssumptionss

1. Randomness & Independence of Errors– Independent Random Samples are Drawn for

each condition

2. Normality– Populations (for each condition) are Normally

Distributed

3. Homogeneity of Variance– Populations (for each condition) have Equal

Variances

About CRDAbout CRD

• Random samples of size n1, n2, …,nk are drawn from k populations with means 1, 2,…, k and with common variance 2.

• Let xij be the j-th measurement (replication) and the i-th sample.

• The total variation in the experiment is measured by the total sum of squarestotal sum of squares:

2)( SS Total xxij 2)( SS Total xxij

ExampleExample

Is the attention duration of children affected by whether or not they had a good breakfast? Twelve children were randomly divided into three groups and assigned to a different meal plan. The response was attention duration in minutes during the morning reading time.

No Breakfast Light Breakfast Full Breakfast

8 (x11) 14 (x21) 10 (x31)

7 (x12) 16 (x22) 12 (x32)

9 (x13) 12 (x23) 16 (x33)

13 (x14) 17 (x24) 15 (x34)

k = 3 treatments. Are the average attention spans different?

k = 3 treatments. Are the average attention spans different?

The ANOVA Table of the CRDThe ANOVA Table of the CRD

MST = SST/(k-1)

MSE = SSE/(n-k)

Source df SS MS F

Treatments k -1 SST SST/(k-1) MST/MSE

Error n - k SSE SSE/(n-k)

Total n -1 Total SS

MS(Mean Squares)

n

xx ij

ij

2

2SSTotal

n

x

n

Tij

i

k

ii 2

1

2

SST

SST SS TotalSSE

SS(Sum of Squares)

Computing Computing FormulasFormulas Note:

CM = Common MeanCM = CFCF =Correction Factor

The ANOVA Table of the CRDThe ANOVA Table of the CRD

The Total SSTotal SS is divided into two parts: SSTSST (sum of squares for treatments): measures the variation among the

k sample means. SSESSE (sum of squares for error): measures the variation within the k

samples.

SSE SST SS Total SSE SST SS Total

• These sums of squaressums of squares behave like the numerator of a sample variance. When divided by the appropriate degrees of freedomdegrees of freedom, each provides a mean squaremean square, an estimate of variation in the experiment.

• Degrees of freedomDegrees of freedom are additive, just like the sums of squares.

dfdfdf Error Trt Total dfdfdf Error Trt Total

The Breakfast ProblemThe Breakfast ProblemNo Breakfast Light Breakfast Full Breakfast

8 14 10

7 16 12

9 12 16

13 17 15

T1 = 37 T2 = 59 T3 = 53 G = 149G = 149

25.58SST-SS TotalSSE

6766.46CM75.1914CM4

53

4

59

4

37SST

122.91671850.0833-1973CM15...78SS Total

0833.185012

149CM

222

222

2


6766.46CM75.1914CM4

53

4

59

4

37SST

122.91671850.0833-1973CM15...78SS Total

0833.185012

149CM

222

222

2


6766.46CM75.1914CM4

59

4

53

4

37SST

122.91671850.0833-1973CM15...78SS Total

0833.185012

149CM

222

222

2


6766.46CM75.1914CM4

59

4

53

4

37SST

122.91671850.0833-1973CM15...78SS Total

0833.185012

149CM

222

222

2

Source df SS MS F

Treatments 2 64.6667 32.3333 5.00

Error 9 58.25 6.4722

Total 11 122.9167

The Breakfast Problem

Testing the Treatment MeansTesting the Treatment Means

Remember that 2 is the common variance for all k populations. The quantity MSE SSE/(n k) is a pooled estimate of 2, a weighted average of all k sample variances, whether or not H0 is true.

versus... :H k3210 different ismean oneleast at :H1

• If H0 is true, then the variation in the sample means, measured by MST [SST/ (k 1)], also provides an unbiased estimate of 2.

• However, if H0 is false and the population means are different, then MST— which measures the variance in the sample means — is unusually large.large. The test statistic F F MST/ MSEMST/ MSE tends to be larger that usual.

The F TestThe F Test

• Hence, you can reject H0 for large values of F,

using a right-tailedright-tailed statistical test.• When H0 is true, this test statistic has an F distribution with

df1 (k 1) and df 2 (n k) degrees of freedom and right-right-

tailedtailed critical values of the F distribution can be used. ... :H 3210 k

. and1 with FF if HReject MSE

MSTF :StatisticTest

0 dfn-k k

Source df SS MS F

Treatments 2 64.6667 32.3333 5.00

Error 9 58.25 6.4722

Total 11 122.9167

.26.4FF :regionRejection

00.54722.6

3333.32

MSE

MSTF

different ismean oneleast at :H

versus:H

.05

1

3210


00.54722.6

3333.32

MSE

MSTF


versus:H

.05

1

3210

The Breakfast ProblemThe Breakfast Problem

We reject H0 and conclude that there is a difference in average of attention spans

Source df SS MS F

Treatments 2 64.6667 32.3333 5.00

Error 9 58.25 6.4722

Total 11 122.9167

Confidence IntervalsConfidence Intervals

.error on based is and MSE where

11)(: Difference

: mean, singleA

22/

2/

dfts

nnstxx

n

stx

jijiji

i

ii

.error on based is and MSE where

11)(: Difference

: mean, singleA

22/

2/

dfts

nnstxx

n

stx

jijiji

i

ii

If a difference exists between the treatment means, we can explore it with confidence intervals.

Tukey’s Method forTukey’s Method forPaired ComparisonsPaired Comparisons

•Designed to test all pairs of population means simultaneously, with an overall overall error rate of error rate of .

•Based on the studentized rangestudentized range, the difference between the largest and smallest of the k sample means.

•Assume that the sample sizes are sample sizes are equalequal and calculate a “ruler” that measures the distance required between any pair of means to declare a significant difference.

different. declared arethey

, than moreby differ means ofpair any If

11. Table from value ),(

size samplecommon

error MSE

means treatmentofnumber where

),( :Calculate

knkq

n

dfdfs

k

n

sdfkq

i

i




size samplecommon

error MSE

means treatmentofnumber where

),( :Calculate

knkq

n

dfdfs

k

n

sdfkq

i

i

Tukey’s MethodTukey’s Method

Use Tukey’s method to determine which of the three population means differ from the others.

02.54

4722.695.3

4)9,3(05.

sq 02.5

4

4722.695.3

4)9,3(05.

sq

No Breakfast Light Breakfast Full Breakfast

T1 = 37 T2 = 59 T3 = 53

Means 37/4 = 9.25 59/4 = 14.75 53/4 = 13.25


List the sample means from smallest to largest.

14.75 13.25 25.9

231 xxx14.75 13.25 25.9

231 xxx 02.5 02.5

Since the difference between 9.25 and 13.25 is less than = 5.02, there is no significant difference. There is a difference between population means 1 and 2 however…There is no difference between 13.25 and 14.75. We can declare a significant difference in average attention spans between “no breakfast” and “light breakfast”, but not between the other pairs.


ExerciseExercise

Method I Method II Method III

48 55 84

73 85 68

51 70 95

65 69 74

237 279 321 837iT

A direct extension of the paired difference or matched pairs design.

A two-way classificationtwo-way classification in which k treatment means are compared.

The design uses blocksblocks of k experimental units that are relatively similar or homogeneous, with one unit within each block randomly assigned to each treatment.

Factor : A A has 4 levels (a1, a2,a3, and a4) Block : 4 blocks (in CRD, this is like replication) So, there fill be 4 x 4 = 16 EU

a4 a4

a4

a4

a2

a2

a2

a2a3

a3

a3

a3

a1

a1

a1

a1

Block1 Block2 Block3 Block4

Let xij be the response for the i-th treatment applied to the j-th block. i = 1, 2, …k j = 1, 2, …, b

The total variation in the experiment is measured by the total sum of squarestotal sum of squares:

2)( SS Total xxij 2)( SS Total xxij

Source df SS MS F


Blocks b -1 SSB SSB/(b-1) MSB/MSE

Error (b-1)(k-1) SSE SSE/(b-1)(k-1)

Total n -1 Total SS

MST = SST/(k-1)

MSB = SSB/(b-1)

MS(Mean Squares)

n

xx ij

ij

2

2SSTotal

n

x

b

Tij

k

ii 2

1

2

SST

SSB-SST SS TotalSSE

SS(Sum of Squares)

n

x

k

B ijj

22

SSB

MSE = SSE/(n-k)

The Total SSTotal SS is divided into 3 parts: SSTSST (sum of squares for treatments): measures

the variation among the k treatment means SSBSSB (sum of squares for blocks): measures the

variation among the b block means SSESSE (sum of squares for error): measures the

random variation or experimental errorin such a way that:

SSE SSB SST SS Total SSE SSB SST SS Total

E B T Total dfdfdfdf E B T Total dfdfdfdf

SSB-SST-SS TotalSSE

block for total whereCMSSB

ent for treatm total whereCMSST

CMSS Total

G whereG

CM

2

2

2

2

jBk

B

iTb

T

x

xn

jj

ii

ij

ij

3333.116667.6138111SSE

6667.6118723

32494536SSB

3818724

486450SST

1112187-10...1511SS Total

218712

621CM

2222

222

222

2

3333.116667.6138111SSE

6667.6118723

32494536SSB

3818724

486450SST

1112187-10...1511SS Total

218712

621CM

2222

222

222

2

Locations

Soil Prep 1 2 3 4 Ti

A 11 13 16 10 50

B 15 17 20 12 64

C 10 15 13 10 48

Bj 36 45 49 32 162

Total df = Mean SquaresTreatment df = Block df =

Error df =

bk –1 = n -1

k –1

bk– (k – 1) – (b-1) = (k-1)(b-1)

MST = SST/(k-1)

MSE = SSE/(k-1)(b-1)

Source df SS MS F


Blocks b -1 SSB SSB/(b-1) MSB/MSE

Error (b-1)(k-1) SSE SSE/(b-1)(k-1)

Total n -1 Total SS

b –1 MSB = SSB/(b-1)

Source df SS MS F

Treatments 2 38 19 10.06

Blocks 3 61.6667 20.5556 10.88

Error 6 11.3333 1.8889

Total 11 122.9167

3333.116667.6138111SSE

6667.6118723

32494536SSB

3818724

486450SST

1112187-10...1511SS Total

218712

621CM

2222

222

222

2

3333.116667.6138111SSE

6667.6118723

32494536SSB

3818724

486450SST

1112187-10...1511SS Total

218712

621CM

2222

222

222

2

ersus v... :H 3210 different ismean oneleast at :H1

For either treatment or block means, we can test:

equal are means block)(or tment that treaH test theTo 0

. )1)(1( and)1 (or 1- with FF if HReject

)MSE

MSBF(or

MSE

MSTF :StatisticTest

0 dfkb bk

If H 0 is false and the population means are different, then MST or MSB— whichever you are testing— will unusually large.large. The test statistic F = MST/ MSEF = MST/ MSE(or F = F = MSB/ MSE)MSB/ MSE) tends to be larger that usual.

We use a right-tailed F test with the appropriate degrees of freedom.

Remember that 2 is the common variance for all bk treatment/block combinations. MSE is the best estimate of 2, whether or not H 0 is true.

Source df SS MS F

Soil Prep (Trts) 2 38 19 10.06

Location (Blocks) 3 61.6667 20.5556 10.88

Error 6 11.3333 1.8889

Total 11 122.9167

n.preparatio soil toduegrowth seedling of difference

a is e that therconclude and Hreject We


06.10MSE

MSTF


versus:H

:npreparatio soil todue difference afor test To

0

.05

a

3210

n.preparatio soil toduegrowth seedling of difference

a is e that therconclude and Hreject We


06.10MSE

MSTF


versus:H

:npreparatio soil todue difference afor test To

0

.05

a

3210

Although not of primary importance, notice that the blocks (locations) were also significantly different (F = 10.88)

.error on based is and MSE

means.block or treatment necessary

theare / and / where

dfts

kBBbTT iiii

.error on based is and MSE

means.block or treatment necessary

theare / and / where

dfts

kBBbTT iiii

If a difference exists between the treatment means or block means, we can explore it with confidence intervals or using Tukey’s method.

bstTT ji

2)( 2

2/Difference in treatmenttreatment means :

Difference in blockblock means :

kstBB ji

2)( 2

2/




error MSE

:where

dfkq

dfdfs




error MSE

:where

dfkq

dfdfs

For comparing treatmenttreatment means :

For comparing blockblock means :

b

sdfkq ),(

k

sdfbq ),(

Use Tukey’s method to determine which of the three soil preparations differ from the others.

98.24

8889.134.4

4)6,3(05.

s

q

98.24

8889.134.4

4)6,3(05.

s

q

A (no prep) B (fertilization) C (burning)

T1 = 50 T2 = 64 T3 = 48

Means 50/4 = 12.5 64/4 = 16 48/4 = 12

List the sample means from smallest to largest.

16.0 12.5 21

BAC TTT16.0 12.5 21

BAC TTT98.2 98.2

Since the difference between 12 and 12.5 is less than = 2.98, there is no significant difference. There is a difference between population means C and B however.

There is a significant difference between A and B.

A significant difference in average growth only occurs when the soil has been fertilized.

A randomized block design should not be used when both treatments and blocks correspond to experimental factors of interest to the researcher

Remember that blocking may not always be beneficial.

Remember that you cannot construct confidence intervals for individual treatment means unless it is reasonable to assume that the b blocks have been randomly selected from a population of blocks.

If the normality assumption is valid, the plot should resemble a straight line, sloping upward to the right.

If not, you will often see the pattern fail in the tails of the graph.

If the normality assumption is valid, the plot should resemble a straight line, sloping upward to the right.

If not, you will often see the pattern fail in the tails of the graph.

Normal Probability Plot

Residual

Perc

ent

3210-1-2-3

99

95

90

80

70

605040

30

20

10

5

1

Normal Probability Plot of the Residuals(response is Growth)

If the equal variance assumption is valid, the plot should appear as a random scatter around the zero center line.

If not, you will see a pattern in the residuals.

If the equal variance assumption is valid, the plot should appear as a random scatter around the zero center line.

If not, you will see a pattern in the residuals.

Residuals versus Fits

Fitted Value

Resi

dual

201816141210

1.5

1.0

0.5

0.0

-0.5

-1.0

-1.5

-2.0

Residuals Versus the Fitted Values(response is Growth)

ExerciseSuppose you wish to compare the means of four populations based on independent random samples, each of which contain 6, 8, 10 and 9 observations respectively. What is the degrees of freedom for the F statistic?

ExerciseComplete the ANOVA table. Source SS df MS FTreatments a 2 5.7 fBlocks 17.1 c e 2.41Error b d 1.42 Total 42.7 17

Documents

Introduction to Probability and Statistics Thirteenth Edition Chapter 11 The Analysis of Variance