Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 1© 2010 Michael Stuart
Lecture 5.1 Part 1"Split Plot" experiments
1. Review of– randomised block designs– hierarchical / nested designs
2. Examples
3. Analysis of Whole Units
4. Analysis of Sub Units
5. Split plot analysis
6. Expected Mean Squares– Error terms for tests
7. Interactions
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 2© 2010 Michael Stuart
Minute Test: How Much
5432
18
16
14
12
10
8
6
4
2
0
How Much
Co
un
t
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 3© 2010 Michael Stuart
Minute Test: How Fast
5432
20
15
10
5
0
How Fast
Co
un
t
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 4© 2010 Michael Stuart
Randomised Blocks, Again
An experiment was conducted to assess the effects of applying four chemicals to soybean seeds with a view to improving germination rates.
Each treatment was applied to 100 seeds planted in adjacent plots. As a check, another plot was planted with 100 seeds which received no treatment.
The experiment was replicated in five blocks of five plots each, with each treatment being assigned to plots at random within each block.
The number of failures in each plot was recorded.
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 5© 2010 Michael Stuart
Testing for Interaction with Blocks
Analysis of Variance for Failures
Source DF Seq SS Adj SS Adj MS F PBlock 4 49.8400 49.8400 12.4600 2.30 0.103Treatment 4 83.8400 83.8400 20.9600 3.87 0.022Block*Treatment 16 86.5600 86.5600 5.4100 **Error 0 * * *Total 24 220.2400
** Denominator of F-test is zero.
Without a valid reference term, it is not possible to have an F test for interaction.
To check for interaction,
– replicate each design point within each block,
– replicates provide estimate of pure error
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 6© 2010 Michael Stuart
Assumption of No Interaction
• With no replication, use block by treatment interaction mean square as error mean square.
• With interaction present, this means
– estimate of is inflated,
– power of the F test for treatment effects is reduced.
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 7© 2010 Michael Stuart
Blocking as Random Effect
Source Expected Mean Square
1 Block (3) + 5.0000 (1)
2 Treatment (3) + Q[2]
3 Error (3)
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 8© 2010 Michael Stuart
Values
Values
•
S
T
eB
eS
B
eT
e = eB + eS + eT
Hierarchy of components of variation
Batchvariation
Samplingvariation
Testingvariation
y
•
•
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 9© 2010 Michael Stuart
Hierarchical Design forEstimating Components of Variation
Batch 1 2 3 4 5 Sample 1 2 3 4 5 6 7 8 9 10 Test 40 39 30 30 26 28 25 26 29 28 14 15 30 31 24 24 19 20 17 17 Batch 6 7 8 9 10 Sample 11 12 13 14 15 16 17 18 19 20 Test 33 32 26 24 23 24 32 33 34 34 29 29 27 27 31 31 13 16 27 24 Batch 11 12 13 14 15 Sample 21 22 23 24 25 26 27 28 29 30 Test 25 23 25 27 29 29 31 32 19 20 29 30 23 23 25 25 39 37 26 28
60 measurementsnested in 30 samplesnested in 15 batches
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 10© 2010 Michael Stuart
Part 2 ExamplesExample 1
3 varieties of wheat are planted in a homogeneous block of 3 plots, with varieties randomly assigned to plots;
the experiment is replicated 4 times, with separate randomisations in each block, as follows:
Block 1 Block 2
V2 V1 V3 V3 V1 V2
Block 3 Block 4
V1 V3 V2 V2 V3 V1
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 11© 2010 Michael Stuart
ALBH AHBL
ALBL AHBH
Example 1
Block 1 Block 2
V2 V1 V3 V3 V1 V2
Block 3 Block 4
V1 V3 V2 V2 V3 V1
Following planting, it was decided to try two new fertilisers. Each plot was divided in four subplots and a 22 was implemented in each, with the possible combinations being assigned at random to subplots within each plot, as shown for one plot below.
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 12© 2010 Michael Stuart
Plot structure
48 subplots
nested in 12 whole plots
nested in 4 blocks
Treatment structure
3 varieties randomly allocated to whole plots within blocks
22 = 4 fertiliser combinations randomly allocated to subplots within whole plots
least variation
in between variation
most variation
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 13© 2010 Michael Stuart
Example 2
Electronic components are baked in an oven at a set temperature for a set time. Two factors thought to influence the life times of the components were the oven temperature and the bake time. Trial settings for these factors were chosen as follows:
Oven Temperature (T), °F, 580, 600, 620, 640,
Baking time (B), min, 5, 10, 15.
To save on costly runs, three components were baked together at each temperature, with one withdrawn at each of the set times. This plan was replicated 3 times. The results follow.
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 14© 2010 Michael Stuart
Example 2
Results of accelerated life time testsfor electronic components
Baking Time (min.)
Replicate Temperature of Oven (°F)
5 10 15
1 580 217 233 175 600 158 138 152 620 229 186 155 640 223 227 156
2 580 188 201 195 600 126 130 147 620 160 170 161 640 201 181 172
3 580 162 170 213 600 122 185 180 620 167 181 182 640 182 201 199
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 15© 2010 Michael Stuart
Example 2
What are the whole units?
What are the whole unit treatments?
What are the sub units?
What are the sub unit treatments?
What is the plot structure?
What is the treatment structure?
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 16© 2010 Michael Stuart
Example 2
What are the whole units?
What are the whole unit treatments?
What are the sub units?
What are the sub unit treatments?
oven load of 3 components
oven temperatures
single components
baking times
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 17© 2010 Michael Stuart
Unit and Treatment Structures
Baking Time (min.)
Replicate Temperature of Oven (°F)
5 10 15
1 580 217 233 175 600 158 138 152 620 229 186 155 640 223 227 156
2 580 188 201 195 600 126 130 147 620 160 170 161 640 201 181 172
3 580 162 170 213 600 122 185 180 620 167 181 182 640 182 201 199
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 18© 2010 Michael Stuart
Unit structure
36 sub units
nested in 12 whole units
nested in 3 blocks
Treatment structure
4 temperatures randomly allocated to whole units within blocks
3 baking times randomly allocated to sub units within whole units
least variation
in between variation
most variation
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 19© 2010 Michael Stuart
Implications of unit and treatment structures
Treatment effects assessed relative to variation between units to which they are applied.
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 20© 2010 Michael Stuart
Case study
Paper manufactured in two stages:
pulp prepared in large batches, long process,
batches divided into small parts, each of which is put through a short cooking process.
Experiment to investigate effects of
three pulp preparation methods,
four cooking temperature settings
on tensile strength of the paper.
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 21© 2010 Michael Stuart
Case study
Protocol:
batch made using one method, randomly selected,
each of four samples "cooked" at one of the four different temperatures, random order
repeated for the other two methods,
replicated on successive days, new random orderings
Results
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 22© 2010 Michael Stuart
Randomised Blocks analysis for Methods
Day Method 1 2 3
1 34.50 35.25 37.25
2 38.75 37.25 39.50
3 31.00 33.25 37.50
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 23© 2010 Michael Stuart
Randomised Blocks analysis for Methods
Analysis of Variance for Y, no interaction term
Source DF Seq SS Adj SS Adj MS F PB 2 19.389 19.389 9.694 4.28 0.102M 2 32.097 32.097 16.049 7.08 0.049Error 4 9.069 9.069 2.267Total 8 60.556
S = 1.50578
Analysis of Variance for Y, with interaction term
Source DF Seq SS Adj SS Adj MS F PB 2 19.3889 19.3889 9.6944 4.28 0.102M 2 32.0972 32.0972 16.0486 7.08 0.049B*M 4 9.0694 9.0694 2.2674 **Error 0 * * *Total 8 60.5556
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 24© 2010 Michael Stuart
Diagnostics
41403938373635343332
2
1
0
-1
-2
-3
Fitted Value
Del
eted
Res
idu
al
Residuals Versus Fitted
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 25© 2010 Michael Stuart
Diagnostics
4
3
2
1
0
-1
-2
-3
-4210-1-2
Del
eted
Res
idu
al
Normal Score
Normal Plot
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 26© 2010 Michael Stuart
Randomised Blocks analysis for Temperature
Day 1 Day 2 Day 3 200 31.00 30.00 32.67 225 34.00 32.67 37.00 250 36.00 38.00 39.67
Temperature
275 38.00 40.33 43.00
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 27© 2010 Michael Stuart
Randomised Blocks analysis for Temperature
Analysis of Variance for Y, no interaction term
Source DF Seq SS Adj SS Adj MS F PB 2 25.879 25.879 12.940 11.30 0.009T 3 144.636 144.636 48.212 42.10 0.000Error 6 6.871 6.871 1.145Total 11 177.386
S = 1.07013
Analysis of Variance for Y, with interaction term
Source DF Seq SS Adj SS Adj MS F PB 2 25.879 25.879 12.940 11.30 0.009T 3 144.636 144.636 48.212 42.10 0.000B*T 6 6.871 6.871 1.145 **Error 0 * * *Total 11 177.386
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 28© 2010 Michael Stuart
Diagnostic analysis
4442403836343230
2
1
0
-1
-2
Fitted Value
Del
eted
Res
idu
alResiduals Versus Fitted
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 29© 2010 Michael Stuart
Diagnostic analysis
3
2
1
0
-1
-2
-3210-1-2
Del
eted
Res
idu
al
Normal Score
Normal Plot
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 30© 2010 Michael Stuart
Response: Strength (Y)
Factors: Day (Block), BMethod, MTemperature, T
Effects to include in model:
BMB*M
TB*TM*T
Split plot analysis
assessed at whole unit level
assessed at sub unit level
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 31© 2010 Michael Stuart
Split plot analysis
Analysis of Variance for Y
Source DF Seq SS Adj SS Adj MS F PB 2 77.556 77.556 38.778 4.68 0.126 xM 2 128.389 128.389 64.194 7.08 0.049B*M 4 36.278 36.278 9.069 2.14 0.138T 3 434.083 434.083 144.694 42.01 0.000B*T 6 20.667 20.667 3.444 0.81 0.580M*T 6 75.167 75.167 12.528 2.96 0.052Error 12 50.833 50.833 4.236Total 35 822.972
x Not an exact F-test.
S = 2.05818
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 32© 2010 Michael Stuart
Split plot analysis
Exercise: Check calculation of F ratios for M and T and corresponding degrees of freedom; cross check with previous analyses.
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 33© 2010 Michael Stuart
4 Expected Mean Squares
Source Expected Mean Square for Each Term
1 B (7) + 3.0000 (5) + 4.0000 (3) + 12.0000 (1)
2 M (7) + 4.0000 (3) + Q[2, 6]
3 B*M (7) + 4.0000 (3)
4 T (7) + 3.0000 (5) + Q[4, 6]
5 B*T (7) + 3.0000 (5)
6 M*T (7) + Q[6]
7 Error (7)
Exercise: Translate into 2 notation.
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 34© 2010 Michael Stuart
Diagnostics
45403530
3
2
1
0
-1
-2
-3
-4
Fitted Value
Del
eted
Res
idu
alResiduals Versus Fitted Values
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 35© 2010 Michael Stuart
Diagnostics
3
2
1
0
-1
-2
-3
-4210-1-2
Del
eted
Res
idu
al
Normal Score
Normal Plot
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 36© 2010 Michael Stuart
Interaction effect?
321
42
40
38
36
34
32
30
Method
Str
eng
th
200225
250275
T
Treatment Profile Plot
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 37© 2010 Michael Stuart
Interaction effect?
275250225200
42
40
38
36
34
32
30
Temperature
Str
eng
th
1
2
3
M
Method Profile Plot
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 38© 2010 Michael Stuart
Reasons for using split plots
• Adding another factor after the experiment started
• Some factors require better precision than others
• Changing one factor is
– more difficult– more expensive– more time consuming
than changing others
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 39© 2010 Michael Stuart
Reading
DCM §4.1, §14.4
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 40© 2010 Michael Stuart
Lecture 5.1 Part 2Further Developments
• Repeated measures
• Robust design
• Analysis of Covariance
• Non-normal error
• Strategies for experiments
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 41© 2010 Michael Stuart
Robust design
Seek optimal settings of experimental factors
that remain optimal,
irrespective of uncontrolled environmental factors.
Run the experimental design, the inner array,at a range of settings of the environmental variables, the outer array.
Popularised by Taguchi.
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 42© 2010 Michael Stuart
Analysis of Covariance
Objective: take account of variation in uncontrolled environmental variables.
Solution: measure the environmental variables at each design point and incorporate in the analysis through regression methods (Analysis of Covariance)
Effects: reduces "error" variation, makes factor effects more significant
adjusts factor effect estimates to take account of extra variation source.
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 43© 2010 Michael Stuart
Analysis of Covariance; Illustration
Breaking strength of monofilament fibreproduced by three different machines,
allowing for variation in fibre thickness.
Machine 1 Machine 2 Machine 3
Y X Y X Y X 36 20 40 22 35 21 41 25 48 28 37 23 39 24 39 22 42 26 42 25 45 30 34 21 49 32 44 28 32 15
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 44© 2010 Michael Stuart
Analysis of Covariance; Minitab
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 45© 2010 Michael Stuart
Analysis of Covariance; Minitab
General Linear Model: Y versus Machine
Source DF Seq SS Adj SS Adj MS F PX 1 305.13 178.01 178.01 69.97 0.000Machine 2 13.28 13.28 6.64 2.61 0.118Error 11 27.99 27.99 2.54Total 14 346.40
S = 1.59505
One-way ANOVA: Y versus Machine
Source DF SS MS F PMachine 2 140.4 70.2 4.09 0.044Error 12 206.0 17.2Total 14 346.4
S = 4.143
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 46© 2010 Michael Stuart
Analysis of Covariance; Minitab
32.530.027.525.022.520.017.515.0
50
45
40
35
30
X
Y
123
Machine
Scatterplot of Y vs X
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 47© 2010 Michael Stuart
Further Developments
• Non-Normal errors
– transformations
– generalised linear models, including Logistic
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 48© 2010 Michael Stuart
Changing spread with log
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 49© 2010 Michael Stuart
Changing spread with log
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 50© 2010 Michael Stuart
Changing spread with log
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 51© 2010 Michael Stuart
Changing spread with log
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 52© 2010 Michael Stuart
Changing spread with log
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 53© 2010 Michael Stuart
Changing spread with log
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 54© 2010 Michael Stuart
Changing spread with log
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 55© 2010 Michael Stuart
Changing spread with log
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 56© 2010 Michael Stuart
Changing spread with log
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 57© 2010 Michael Stuart
Why the log transform works
High spread at high X
transformed to
low spread at high Y
Low spread at low X
transformed to
high spread at low Y
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 58© 2010 Michael Stuart
Strategies for Experimenting
– Consultation
– Planning
– Resources
– Ethical issues
– Implementation of design
– Application of results
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 59© 2010 Michael Stuart
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 60© 2010 Michael Stuart
Strategy for ExperimentationShewhart's PDCA Cycle
Check
Act
Plan
Do
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 61© 2010 Michael Stuart
Strategy for ExperimentationShewhart's PDCA Cycle
• Plan: Plan a change to the process, predict its effect, plan to measure the effect
• Do: Implement the change as an experiment and measure the effect
• Check: Analyse the results to learn what effect the change had, if any
• Act: If successful, make the change permanent, proceed to plan the next improvement
or
if not, proceed to plan an alternative change
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 62© 2010 Michael Stuart
Strategy for Experimentation:new vs old manufacturing process
Plan:
• Compare defect rates for old process and new (cheaper) process
– predict reduction, or no increase, in number of defectives using new process
• Sample output over an eight week period, six days per week
– select 50 components at random per day
• Count number of defectives per sample
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 63© 2010 Michael Stuart
Do:
• Implement plan
• Record daily numbers of defectives
Assessing experimental process for manufacturing electronic components
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 64© 2010 Michael Stuart
Check:
• Analyse data
• test statistical significance of the change
Assessing experimental process for manufacturing electronic components
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 65© 2010 Michael Stuart
Act:
• If no worse, make the change permanent,
– proceed to plan the next improvement
or
• if not, proceed to plan an alternative change
Assessing experimental process for manufacturing electronic components
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 66© 2010 Michael Stuart
Resources
e.g. sample size
Need to know
Also, need to know €
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 67© 2010 Michael Stuart
Ethical issues
– withholding medical treatment?
– double-blind experiments,
– inadequate budget puts patients at risk for non-informative results
Diploma in StatisticsDesign and Analysis of Experiments
Lecture 5.1 68© 2010 Michael Stuart
Strategy
When you see the credits roll at the end of a successful movie you realize there are many more things that must be attended to in addition to choosing a good script.
Similarly in running a successful experiment there are many more things that must be attended to in addition to choosing a good experimental design.
Ref: Robinson, G.K., Practical Strategies for Experimenting, Wiley, 2000.