Modern Experimental Design Some developments and applications

Emlyn WilliamsStatistical Consulting Unit

The Australian National University

Block1 2 3 4 5 6 71 2 3 4 5 6 72 3 4 5 6 7 14 5 6 7 1 2 3

A Balanced Incomplete Block design

v=7 treatments, r=3 replications, k=3 plots per block

3111111131111111311111113111111131111111311111113

NNConcurrence matrix

Information Matrix NNk

rIA 1

Average Efficiency Factor)(

1

AtracervE

Some design properties

)1()1(

vk

vkE(Balanced Incomplete Block Design)

A Balanced Square Lattice design for 9 varieties

Replicate 1 Replicate 2Block 1 2 3 1 2 3

____________ ___________1 4 7 1 9 52 5 8 6 2 73 6 9 8 4 3

Replicate 3 Replicate 4Block 1 2 3 1 2 3

___________ ___________1 7 4 1 2 35 2 8 4 5 69 6 3 7 8 9

α - Designs• They are a type of incomplete block design• Generated from α - arrays• One-dimensional blocking structure• Available for a wide range of parameter sets• Quickly generate efficient designs, especially

for large variety numbers• Good for nested variety structures• Construct designs in CycDesigN

6

Unrandomized α-design for 20 varieties

0 0 0 00 1 2 30 2 4 10 3 1 4

Rep 11 2 3 4 56 7 8 9 10

11 12 13 14 1516 17 18 19 20

Rep 21 2 3 4 57 8 9 10 6

13 14 15 11 1219 20 16 17 18

Rep 31 2 3 4 58 9 10 6 7

15 11 12 13 1417 18 19 20 16

Rep 41 2 3 4 59 10 6 7 8

12 13 14 15 1120 16 17 18 19

α-Array

Replicate

Plot

α(0,1)-design

k'=4, s=5 v=k's

α –Design 1D - blocking 19 varieties

Replicate 1

Replicate 2

Replicate 3

1617 2

7 1

8 15 6 3

1211

1914

13

10 9

18 4

5

Row-column designs

• Two-dimensional blocking structures• Strata

– replicates– rows within replicates– columns within replicates– plots

• Better field control than incomplete blocks• Latinized designs if replicates are together

9

Latinized row-column design C o l u m n 1 2 3 4 5 1 6 1 7 1 9 1 8 1 3 6 2 4 8 1 5 R e p 1 1 2 5 7 1 0 1 1 2 0 9 1 4 1 3 5 1 5 1 1 9 2 0 4 1 2 1 7 3 8 R e p 2 1 3 1 8 6 9 7 1 0 1 4 1 1 1 6 2 1 5 7 3 1 7 1 0 1 9 1 2 1 3 1 2 R e p 3 8 1 6 5 1 4 6 1 1 4 1 8 2 0 9

2-Latinized row-column design for 18 varieties

C o l u m n1 2 3 4 5 6

1 5 3 1 8 9 1 6 1 05 1 3 7 6 1 7 1 2 R e p 11 2 8 4 1 1 1 49 1 7 2 1 1 1 8 1 37 1 0 1 6 5 1 8 R e p 2

1 4 4 1 2 3 6 1 51 6 1 8 1 3 1 4 4 5

6 1 1 1 0 1 5 7 2 R e p 38 1 2 1 1 7 3 9

Some other design types

– Factorial Designs

– Crossover Designs

– Spatial Designs

– Partially-replicated Designs

Glasshouse experiment

–3 factors

• Plant genotype (6 levels)• Nitrate (4 levels)• Bacterial strain (2 levels)

–6 benches (replicates) 12 x 4 pots

Glasshouse layout of factorial experiment

4 pots12 pots1

2

43

5

6

12 pots 12 pots

4 pots4 pots

Part of CycDesigN log file• Design parameters• Factors = 6 x 4 x 2• Number of rows = 12• Number of columns = 4• Number of replicates = 6• Random number seed for design generation = 657• Factorial design average efficiency factors• Effect Ave Efficiency Factors (Upper bounds)• E1 0.880702 (0.899971)• E2 1.000000 (1.000000)• E3 1.000000 (1.000000)• E12 0.555185 • E13 0.654721 • E23 0.569515

Replicate 1 of the 6x4x2 factorial design• column 1 2 3 4• row +----------------------------------------------------• 1 | (5,2,1) (1,1,2) (6,3,1) (3,4,2)• 2 | (1,3,2) (4,2,1) (2,1,2) (6,4,1)• 3 | (4,4,1) (5,1,2) (3,2,2) (2,3,1)• 4 | (5,3,2) (2,4,1) (6,2,2) (4,1,1)• 5 | (2,2,1) (1,3,1) (5,4,2) (4,1,2)• 6 | (6,1,1) (2,4,2) (4,3,2) (3,2,1)• 7 | (6,1,2) (3,4,1) (4,3,1) (1,2,2)• 8 | (1,4,1) (6,3,2) (3,1,1) (5,2,2)• 9 | (3,1,2) (6,2,1) (1,4,2) (5,3,1)• 10 | (3,3,1) (4,2,2) (2,1,1) (6,4,2)• 11 | (2,2,2) (3,3,2) (5,4,1) (1,1,1)• 12 | (4,4,2) (5,1,1) (1,2,1) (2,3,2)

Williams 6 x 6 Crossover DesignOrder Subject

1 1 2 3 4 5 62 3 4 5 6 1 23 2 3 4 5 6 14 5 6 1 2 3 45 6 1 2 3 4 56 4 5 6 1 2 3

Williams 6 x 6 Crossover DesignOrder Subject

1 1 2 3 4 5 62 31 42 53 64 15 26

3 23 34 45 56 61 12

4 52 63 14 25 36 41

5 65 16 21 32 43 54

6 46 51 62 13 24 35

• Investigation of respiratory failure• 13 subjects (babies)• 4 doses of nitric oxide• Variate is post-ductal arterial oxygen

tension (pco2resp)

13 x 4 dose-response study(Jones and Kenward Ex 5.1)

Design of dose-response study

Period 1 2 3 4 5 6 7 8 9 10 11 12 13

1 D B C A D C C C A B A C C2 C C D B C A D B B D C A B3 B A B C A B A D D A B B A4 A D A D B D B A C C D D D

Subject

Treatments (Doses of nitric oxide)A – 5ppmB – 10ppmC – 20ppmD – 40ppm

SAS Mixed output for analysis of 13x4 dose-response study

Type 1 Tests of Fixed Effects

Num DenEffect DF DF F Value Pr > F

SUBJECT 12 27 10.73 <.0001PERIOD 3 27 1.23 0.3172DOSE 3 27 0.53 0.6636CARRY1 3 27 1.06 0.3804

Covariance ParameterEstimates

Cov Parm Estimate

Residual 2.9614

Crossover design from CycDesigN Correlated error structure

0.117 0.117 0.117 0.122 0.114

0.119AR (0.95):

0.113 0.113 0.113 0.118 0.110

0.115LV:

Period 1 2 3 4 5 6 7 8 9 10 11 12 131 B D B C A B A C C A D C D2 D B C A D A D B D B C A C3 A A D B B C C D A C A D B4 C C A D C D B A B D B B A

Subject

Pairwise variances

•Jones, B. and Kenward, M.G. (2003). Design and Analysis of Cross-Over Trials, 2nd edn. London: Chapman and Hall.•Williams, E.J. (Experimental designs balanced for the estimation of residual effects of treatments. Austral. J. Scientific Res. 2, 149-168.•Williams, E.R. and John, J.A. (2007). Construction of crossover designs with correlated errors. Austral. and N.Z. J, Statist. 49, 61-68.

Some Crossover design and analysis references

……. …….

Incomplete Block Model

A replicate

Pairwise variance between two plots

within a block =

between blocks =

22

Block 1 Block 2 Block 3

)( 222 b

……. …….

Linear Variance plus Incomplete Block Model

A replicate

Pairwise variance between two plots

within a block =

between blocks =

)(2 212 jj

Block 1 Block 2 Block 3

)( 222 b

k

Distance

Semi VariogramsVariance

k

Distance

Variance

2

22b

2

22b

IB

LV+IB

Sugar beet trials•174 sugar beet trials

•6 different sites in Germany 2003 – 2005

•Trait is sugar yield

•10 x 10 lattice designs

•Three (2003) or two (2004 and 2005) replicates

•Plots in array 50x6 (2003) or 50x4 (2004 and 2005)

•Plots 7.5m across rows and 1.5m down columns

•A replicate is two adjacent columns

•Block size is 10 plots

Selected model type: 2003 2004 2005Baseline (row+column+nugget)

1 3 5

Baseline + IAR(1) 7 6 5Baseline + AR(1)AR(1) 24 6 7Baseline + ILV 4 11 8Baseline + LV+LV 4 8 14Baseline + JLV 0 8 4Baseline + LVLV 20 18 11Total number of trials 60 60 54Median of parameter estimates for AR(1)AR(1) model:Median R 0.94 0.93 0.92Median C 0.57 0.34 0.35Median % nugget§ 25 47 37

§ Ratio of nugget variance over sum of nugget and spatial variance

Sugar beet trials - Number of times selected

Sugar beet trials- 1D analysesNumber of times selected

Selected model type: 2003 2004 2005

Baseline (repl+block+nugget)

17 38 29

Baseline + AR(1) in blocks 7 2 3

Baseline + LV in blocks 36 20 22

Total number of trials 60 60 54

Median of parameter estimates for AR(1) model

Median 0.93 0.93 0.82

Median % nugget§ 36 54 53

§ Ratio of nugget variance over sum of nugget and spatial variance

•Baseline model is often adequate•Spatial should be an optional add-on•One-dimensional spatial is often adequate for thin plots•Spatial correlation is usually high across thin plots•AR correlation can be confounded with blocks•LV compares favourably with AR when spatial is needed•LV allows randomization protection

Observations

Randomization in the Design of Experiments

•“Yates (1939) at the conclusion of the discussion (between Student and Fisher on the merits or randomized vs systematic designs), suggested that, while there might sometimes be small gains in precision to be achieved by systematic arrangements, the lack of security in the basis for error estimation in such designs detracted attention from key issues of interpreting the effects under study. •More recent work in a similar vein, stemming from Bartlett (1978) and Wilkinson et al. (1983) has been based on explicit time series or spatial models of variability, often leading to the so-called neighbourhood balance designs. Again, however, the reality of any apparent gain in precision depends on the adequacy of the assumed model.”

•D.R. Cox (2009) International Statistical Review

•Papadakis, J.S. (1937). Méthode statistique pour des expériences sur champ. Bull. Inst. Amél.Plantes á Salonique 23.•Wilkinson, G.N., Eckert, S.R., Hancock, T.W. and Mayo, O. (1983). Nearest neighbour (NN) analysis of field experiments (with discussion). J. Roy. Statist. Soc. B45, 151-211.•Williams, E.R. (1986). A neighbour model for field experiments. Biometrika 73, 279-287.•Gilmour, A.R., Cullis, B.R. and Verbyla, A.P. (1997). Accounting for natural and extraneous variation in the analysis of field experiments. JABES 2, 269-293.•Williams, E.R., John, J.A. and Whitaker. D. (2006). Construction of resolvable spatial row-column designs. Biometrics 62, 103-108.•Piepho, H.P., Richter, C. and Williams, E.R. (2008). Nearest neighbour adjustment and linear variance models in plant breeding trials. Biom. J. 50, 164-189.•Piepho, H.P. and Williams, E.R. (2010). Linear variance models for plant breeding trials. Plant Breeding 129, 1-8.

Some Spatial design and analysis references

32

Federer augmented lattice square design

1 x x 8 1 x x 7 1 x x 6 1 x x 55 2 x x 8 2 x x 7 2 x x 6 2 x xx 6 3 x x 5 3 x x 8 3 x x 7 3 xx x 7 4 x x 6 4 x x 5 4 x x 8 4

Replicate1 2 3 4

12300000

C

01230000

R

Columns generated by

Rows generated by

Williams, E.R. and John, J.A. (2003). A note on the construction of unreplicated trials. Biom. J. 45.

Constraintsapply

33

P-rep design for 20 varieties

0 0 0 00 1 2 30 2 4 10 3 1 4

Location 11 2 3 4 56 7 8 9 10

11 12 13 14 1516 17 18 19 20

1 2 3 4 57 8 9 10 6

13 14 15 11 1219 20 16 17 18

Location 21 2 3 4 58 9 10 6 7

15 11 12 13 1417 18 19 20 16

1 2 3 4 59 10 6 7 8

12 13 14 15 1120 16 17 18 19

α-Array

Replicate

Plot

1 1 1 01 1 0 11 0 1 10 1 1 1

Drop array

Some drop arrays for v=36 varieties• Number of locations: c=3• Number of units per location: u=48• Block size of p-rep design: k=4• Number of replications in p-rep design: r=(uc/v)=4

• Block size of replicated varieties: m=2, 3

34

1 1 1 0 1 01 1 0 1 0 11 0 1 0 1 10 1 0 1 1 11 0 1 1 1 00 1 1 1 0 1

0 0 1 1 1 11 1 1 1 0 01 0 1 0 1 10 1 1 1 1 01 1 0 0 1 11 1 0 1 0 1

m=2 m=3

associatedα -arrayk'=6, s=6

P-rep design for 36 varieties

35

8 9 10 11 12 7 11 12 7 8 9 1018 13 14 15 16 17 21 22 23 24 19 2029 30 25 26 27 28 28 29 30 25 26 2735 36 31 32 33 34 31 32 33 34 35 36

6 1 2 3 4 5 1 2 3 4 5 68 9 10 11 12 7 7 8 9 10 11 12

15 16 17 18 13 14 20 21 22 23 24 1920 21 22 23 24 19 32 33 34 35 36 31

1 2 3 4 5 6 2 3 4 5 6 117 18 13 14 15 16 13 14 15 16 17 1824 19 20 21 22 23 25 26 27 28 29 3026 27 28 29 30 25 32 33 34 35 36 31

c=3, u=48, k=4, r=4, m=3Block

0 0 1 1 1 11 1 1 1 0 01 0 1 0 1 10 1 1 1 1 01 1 0 0 1 11 1 0 1 0 1

P-rep algorithm• Input v varieties, c locations, u units per location• Calculate replication, r of p-rep design• Choose block size, k and block size of replicated

varieties, m• Generate drop array• Optimize α-array (using modified update procedure

for the average efficiency factor E)• Calculate upper bounds (modified block design

calculations)• Row-column optimization

36

Some other examplesDesign 1 Design 2

Number of varieties v 2500 180Number of locations c 4 5Number of units per location u 3125 216Block size (s) of p-rep design k1 , k2 62 , 63 12Number of replications in p-rep design r=(uc/v)

5 6

Block size of replicated varieties m 25 4Drop array size k' x 2c 100 x 8 20 x 10Efficiency factor of p-rep design E 0.98084 0.91220Efficiency factor upper bound U 0.98085 0.91277

37

CycDesigN 4.0•Windows 95 to XP, Vista and 7•Visual C++•Resolvable / non-resolvable•Block / row-column•One / two stage•Cyclic / alpha / other•Factorial / nested treatments•t-Latinized / partially-latinized•Unequal block sizes•Crossover designs•Spatial designs•Unreplicated designs•P-rep designs (Version 5)

http://www.vsni.co.uk/

Some Unreplicated design references•Federer, W. T. (2002). Construction and analysis of an augmented lattice square design. Biometrical Journal 44, 241-257. •Williams, E. R. and John, J. A. (2003). A note on the design of unreplicated trials. Biometrical Journal 45, 751-757•Cullis, B. R., Smith, A. B. and Coombes, N. E. (2006). On the design of early generation variety trials with correlated data. Journal of Agricultural, Biological and Environmental Statistics 11, 381-393.•Williams, E. R., Piepho, H-P. and Whitaker, D. (2011). Augmented p-rep designs Biometrical Journal 53, 19-27.