Center for Biofilm Engineering Al Parker, PhD, Biostatistician Center for Biofilm Engineering Montana State University Statistics and Biofilms June 29,

Center for Biofilm Engineering

Al Parker, PhD, BiostatisticianCenter for Biofilm EngineeringMontana State University

Statistics and Biofilms

June 29, 2012

Standardized Biofilm Methods Laboratory

Darla GoeresAl Parker

Marty Hamilton

Diane Walker

Lindsey Lorenz

Paul Sturman

Kelli Buckingham-Meyer

What is statistical thinking?

Data

Experimental Design

Uncertainty and variability assessment

What is statistical thinking?

Data (pixel intensity in an image? log(cfu) from viable plate counts?)

Experimental Design - controls - randomization- replication (How many coupons?

experiments? technicians? labs?)

Uncertainty and variability assessment

Why statistical thinking?

Anticipate criticism (design method and experiments accordingly)

Provide convincing results (establish statistical properties)

Increase efficiency (conduct the least number of experiments)

Improve communication

Why statistical thinking?

Standardized Methods

Attributes of a Standard Method: Seven R’s

Relevance

Reasonableness

Resemblance

Repeatability (intra-laboratory)

Responsiveness

Ruggedness

Reproducibility (inter-laboratory)


Relevance

Reasonableness

Resemblance


Responsiveness

Ruggedness


A standard laboratory method is said to be relevant to a real-world scenario if, given the same inputs, the laboratory outcome is equivalent to the real-world outcome.

Relevance

Elbow Prosthesis - in vivo study

Urinary catheter in vivo study

Urinary Catheter Biofilm

CV Catheter in vivo study

Biofilm in the Catheter Tip

1,000 X magnification Sheep (control)


Relevance

Reasonableness

Resemblance


Responsiveness

Ruggedness


A standard laboratory method is said to be reasonable if the method can be performed inexpensively using typical microbiological techniques and equipment.

Reasonableness


Relevance

Reasonableness

Resemblance


Responsiveness

Ruggedness


Resemblance of Controls

Independent runs of the method produce nearly the same control data, as indicated by a small

standard deviation.

Statistical tool:

nested analysis of variance (ANOVA)

• 86 mm x 128 mm plastic plate with 96 wells• Lid has 96 pegs

Resemblance Example: MBEC

1 2 3 4 5 6 7 8 9 10 11 12

A 100 100 100 100 100 50:N N GC SC

B 50 50 50 50 50 50:N N GC SC

C 25 25 25 25 25 50:N N GC SC

D 12.5 12.5 12.5 12.5 12.5 50:N N GC

E 6.25 6.25 6.25 6.25 6.25 50:N N GC

F 3.125 3.125 3.125 3.125 3.125 50:N N GC

G 1.563 1.563 1.563 1.563 1.563 50:N N GC

H 0.781 0.781 0.781 0.781 0.781 50:N N GC

MBEC Challenge Plate

disinfectant neutralizer test control


Mean LD= 5.55

Control Data: log10(cfu/mm2) from viable plate counts

row cfu/mm2 log(cfu/mm2)A 5.15 x 105 5.71B 9.01 x 105 5.95C 6.00 x 105 5.78D 3.00 x 105 5.48E 3.86 x 105 5.59F 2.14 x 105 5.33G 8.58 x 104 4.93H 4.29 x 105 5.63

Exp RowControl

LDMean

LD SD1 A 5.71

5.55 0.311 B 5.951 C 5.781 D 5.481 E 5.591 F 5.331 G 4.931 H 5.63

2 A 5.41

5.41 0.172 B 5.712 C 5.542 D 5.332 E 5.112 F 5.482 G 5.332 H 5.41


Resemblance from experiment to experiment

Mean LD = 5.48

Sr = 0.26

the typical distance between a control well LD from an experiment and the true mean LD

Resemblance from experiment to experiment

The variance Sr2

can be partitioned:

98% due to among experiment sources

2% due to within experiment sources

S

nc • m

c2

+

Formula for the SE of the mean control LD, averaged over experiments

Sc = within-experiment variance of control LDs

SE = among-experiment variance of control LDs

nc = number of control replicates per experiment

m = number of experiments

2

2

S

m

E2

SE of mean control LD =

CI for the true mean control LD = mean LD ± tm-1 x SE

8 • 2

Formula for the SE of the mean control LD, averaged over experiments

Sc = 0.02 x (0.26)2 = 0.00124

SE = 0.98 x (0.26)2 = 0.06408

nc = 8

m = 2

2

2

2SE of mean control LD =

0.00124+

0.06408= 0.1792

95% CI for the true mean control LD = 5.48 ± 12.7 x 0.1792

= (3.20, 7.76)

Resemblance from technician to technician

Mean LD = 5.44

Sr = 0.36

the typical distance between a control well LD and the true mean LD

The variance Sr2

can be partitioned:

0% due to technician sources

24% due to between experiment sources

76% due to within experiment sources

Resemblance from technician to technician


Relevance

Reasonableness

Resemblance


Responsiveness

Ruggedness


Repeatability

Independent runs of the method in the same laboratory produce nearly the same outcome, as indicated by a small

repeatability standard deviation.

Statistical tool: nested ANOVA

Repeatability Example

Data: log reduction (LR)

LR = mean(control LDs) – mean(disinfected LDs)

Exp RowControl

LDMean

LD SD1 A 5.71

5.55 0.311 B 5.951 C 5.781 D 5.481 E 5.591 F 5.331 G 4.931 H 5.63

2 A 5.41

5.41 0.172 B 5.712 C 5.542 D 5.332 E 5.112 F 5.482 G 5.332 H 5.41

Repeatability Example: MBEC

1 2 3 4 5 6 7 8 9 10 11 12A 100 100 100 100 100 50:N N GC SC

B 50 50 50 50 50 50:N N GC SC

C 25 25 25 25 25 50:N N GC SC

D 12.5 12.5 12.5 12.5 12.5 50:N N GC

E 6.25 6.25 6.25 6.25 6.25 50:N N GC

F 3.125 3.125 3.125 3.125 3.125 50:N N GC

G 1.563 1.563 1.563 1.563 1.563 50:N N GC

H 0.781 0.781 0.781 0.781 0.781 50:N N GC

Repeatability Example: MBEC

Mean LR = 1.63

Exp RowControl

LDControl

Mean LD ColDisinfected 6.25% LD

Disinfected Mean LD LR

1 A 5.71

5.55 4.51 1.04

1 B 5.95 1 4.671 C 5.78 2 4.411 D 5.48 3 4.331 E 5.59 4 4.591 F 5.33 5 4.541 G 4.931 H 5.63

2 A 5.41

5.41 3.20 2.21

2 B 5.71 1 4.782 C 5.54 2 2.712 D 5.33 3 3.482 E 5.11 4 3.232 F 5.48 5 1.822 G 5.332 H 5.41

Repeatability Example

Mean LR = 1.63

Sr = 0.83

the typical distance between a LR for an experiment and the true mean LR

S

nc • m

c2

+

Formula for the SE of the mean LR, averaged over experiments


Sd = within-experiment variance of disinfected LDs

SE = among-experiment variance of LRs


nd = number of disinfected replicates per experiment


2

2

2

S

nd • m

d2

+S

m

E2

SE of mean LR =



Sd = within-experiment variance of disinfected LDs

SE = among-experiment variance of LRs


nd = number of disinfected replicates per experiment


2

2

2

CI for the true mean LR = mean LR ± tm-1 x SE


Sc2 = 0.00124

Sd2 = 0.47950

SE2 = 0.59285

nc = 8, nd = 5, m = 2

SE of mean LR =

8 • 2 2

0.00124+

0.59285

5 • 2

0.47950+ = 0.5868

95% CI for the true mean LR = 1.63 ± 12.7 x 0.5868

= 1.63 ± 7.46

= (0.00, 9.09)

How many coupons? experiments?

nc • m m

0.00124+

0.59285

nd • m

0.47950+margin of error= tm-1 x

no. control coupons (nc): 2 3 5 8 12no. disinfected coupons (nd): 2 3 5 5 12

no. experiments (m) 2 8.20 7.80 7.46 7.46 7.163 2.27 2.15 2.06 2.06 1.974 1.45 1.38 1.32 1.32 1.276 0.96 0.91 0.87 0.87 0.84

10 0.65 0.62 0.59 0.59 0.57100 0.18 0.17 0.16 0.16 0.16


Relevance

Reasonableness

Resemblance


Responsiveness

Ruggedness


A method should be sensitive enough that it can detect important changes in parameters of interest.

Statistical tool: regression and t-tests

Responsiveness


Responsiveness Example: MBEC

A: High Efficacy

H: Low Efficacy

1 2 3 4 5 6 7 8 9 10 11 12

A 100 100 100 100 100 50:N N GC SC

B 50 50 50 50 50 50:N N GC SC

C 25 25 25 25 25 50:N N GC SC

D 12.5 12.5 12.5 12.5 12.5 50:N N GC

E 6.25 6.25 6.25 6.25 6.25 50:N N GC

F 3.125 3.125 3.125 3.125 3.125 50:N N GC

G 1.563 1.563 1.563 1.563 1.563 50:N N GC

H 0.781 0.781 0.781 0.781 0.781 50:N N GC


This response curve indicates responsiveness to decreasing efficacy between rowsC, D, E and F


Responsiveness can be quantified with a regression line:

LR = 6.08 - 0.97row

For each step in the decrease of disinfectant efficacy, the LR decreases on average by 0.97.


Relevance

Reasonableness

Resemblance


Responsiveness

Ruggedness


A standard laboratory method is said to be rugged if the outcome is unaffected by slight departures from the protocol.

Ruggedness

Parameters in the protocol:

Sonication Power: 130, 250, 480 watts

Sonication Duration: 25, 30, 35 minutes

Treatment Temperature: 20, 22, 24 oC

Incubation Time: 16, 17, 18 hours

Ruggedness Testing of the MBEC

Ruggedness Test Design

Run Incubation Time

TreatmentTemperature

Sonication Power

Sonication Duration

1 17 hrs 22°C 250 watts 30

2 18 hrs 20°C 130 watts 25

3 16 hrs 24°C 480 watts 35

4 18 hrs 24°C 480 watts 25

5 18 hrs 24°C 130 watts 35

6 18 hrs 20°C 480 watts 35

7 16 hrs 20°C 480 watts 25

8 17 hrs 22°C 250 watts 30

9 16 hrs 20°C 130 watts 35

10 16 hrs 24°C 130 watts 25

1 2 3 4 5 6 7 8 9 10 11 12

A 100 100 100 100 100 50:N N GC SC

B 50 50 50 50 50 50:N N GC SC

C 25 25 25 25 25 50:N N GC SC

D 12.5 12.5 12.5 12.5 12.5 50:N N GC

E 6.25 6.25 6.25 6.25 6.25 50:N N GC

F 3.125 3.125 3.125 3.125 3.125 50:N N GC

G 1.563 1.563 1.563 1.563 1.563 50:N N GC

H 0.781 0.781 0.781 0.781 0.781 50:N N GC



Ruggedness Testing of the Controls

TempSonDurPower

Inc

2422203525303525

480130480130250480130480130161818161718161618

5.75

5.50

5.25

5.00

4.75

4.50

4.25

4.00

me

an lo

g d

en

sity

Ruggedness Testing of the Controls

LD(controls) = 5.027 + 0.1111(IncubationTime – 17) - 0.0042(SonicationDuration -30) - 0.1178(TreatmentTemperature – 22) + 0.0004(SonicationPower – 250) + 0.3893(BiofilmGrowth – 5.87) All terms are small and not of practical importance inside the range of values tested

None of the model terms were statistically significant

This model allows one to quantifiably predict how deviations from the protocol affect the experimental outcome

1 2 3 4 5 6 7 8 9 10 11 12

A 100 100 100 100 100 50:N N GC SC

B 50 50 50 50 50 50:N N GC SC

C 25 25 25 25 25 50:N N GC SC

D 12.5 12.5 12.5 12.5 12.5 50:N N GC

E 6.25 6.25 6.25 6.25 6.25 50:N N GC

F 3.125 3.125 3.125 3.125 3.125 50:N N GC

G 1.563 1.563 1.563 1.563 1.563 50:N N GC

H 0.781 0.781 0.781 0.781 0.781 50:N N GC



Ruggedness Testing of the LRs

plate

row

6

5

4

3

2

1

0

log

re

du

ctio

n


LR(H) = 0.2157 – 0.3738(IncubationTime – 17)* + 0.0015(SonicationDuration -30) – 0.1001(TreatmentTemperature – 22) + 0.0003(SonicationPower – 250)

Only IncubationTime was statistically significant*

Except for IncubationTime, the terms are small and not of practical importance inside the range of values tested


Only IncubationTime was statistically significant*

Except for TreatmentTemperature, the terms are small and not of practical importance inside the range of values tested

LR(A) = 5.7219 + 0.1254(IncubationTime – 17)* + 0.0015(SonicationDuration -30) – 0.2831(TreatmentTemperature – 22) + 0.0003(SonicationPower – 250)

Results of the ASTM ILS for the MBEC

Relevance

Reasonableness

Resemblance


Responsiveness

Ruggedness


Collaboration

http://www.montana.edu/

Collaboration

http://www.montana.edu/

http://www.epa.gov/

http://www.ansi.org/

ASTM Interlaboratory Study (ILS) Process

Register test method

Conduct ruggedness testing

Minimum of 6 participating labs

Technical contact• Instructions• Supplies• Data template

Research report

Precision & Bias statement

ASTM ILS #25570

Eight labs

Three experimental test days at each lab

Three disinfectants tested/experiment day• non-chlorine oxidizer• phenol• quaternary ammonium compound


Relevance

Reasonableness

Resemblance


Responsiveness

Ruggedness


Control Data

Control Data

Untreated Control Variability

LabNo Exp

Mean LD

Within plate

%

Among plate

%

Among exp day

%

Among lab

%

Repeatability SD

Reproducibility SD

1 3 7.50 40% 34% 25% 0.1369 2 3 7.58 20% 27% 53% 0.4206 3 3 6.27 39% 12% 49% 0.1696 4 3 7.92 17% 0% 83% 0.2315 5 3 7.80 64% 0% 36% 0.1624 6 3 7.72 8% 7% 85% 0.5301 7 3 8.13 76% 24% 0% 0.1438 8 3 8.16 51% 0% 49% 0.2706

All 24 7.48 4% 11% 9% 76% 0.3252 0.6669


Relevance

Reasonableness

Resemblance


Responsiveness

Ruggedness


Independent runs of the method by different researchers in different laboratories produce nearly the same outcome (e.g. LR).

This assessment requires a collaborative (multi-lab) study.

Reproducibility

Treated Data: LR (Non-chlorine oxidizer)

Treated Data: LR (Phenol)

Treated Data: LR (Quat)

Oxidizer Results

Disinfectant Row Mean LRWithin

Among lab %

Repeatability SD

Reproducibility SDlab %

Oxidizer

A 5.50 75% 25% 1.0557 1.2205B 4.41 96% 4% 1.4918 1.5196C 3.03 92% 8% 1.6093 1.6771D 1.72 85% 15% 1.5658 1.6986E 0.60 50% 50% 0.8844 1.2488F -0.08 34% 66% 0.3776 0.6453G -0.19 100% 0% 0.4687 0.4687H -0.18 100% 0% 0.5223 0.5223

Phenol Results


Among lab %

Repeatability SD


Phenol

A 5.64 100% 0% 1.2578 1.2578B 4.76 100% 0% 1.2747 1.2747C 2.59 80% 20% 1.2467 1.3979D 1.15 57% 43% 0.8984 1.1905E 0.34 29% 71% 0.326 0.6059F -0.02 52% 48% 0.2521 0.3509G -0.11 56% 44% 0.2015 0.2683H -0.15 100% 0% 0.3009 0.3009

Quat Results


Among lab %

Repeatability SD


Quat

A 3.64 36% 64% 0.9036 1.512B 2.26 35% 65% 0.862 1.4522C 1.34 46% 54% 0.8372 1.2406D 0.95 27% 73% 0.606 1.1715E 0.58 26% 74% 0.5302 1.0394F 0.18 50% 50% 0.3901 0.5501G -0.01 78% 22% 0.3944 0.4472H -0.11 100% 0% 0.3598 0.3598

Repeatability at a glance …

6543210

1.6

1.4

1.2

1.0

0.8

0.6

0.4

0.2

0.0

Mean LR

Re

pe

atab

ility

SD

OxidizerPhenol.Quat.

Dis.

Repeatability SD vs lab means

Reproducibility at a glance …

6543210

1.8

1.6

1.4

1.2

1.0

0.8

0.6

0.4

0.2

Mean LR

Re

pro

du

cib

ility

SD


Dis.

Reproducbility SD versus lab means

ASTM Precision and Bias Statement

Untreated Control Data Variance Assessment

# of Labs

# of Exps

Mean LDa

Sources of Variability

Repeatability SDb

Reproducibility SDb

Within plate

%

Amongplate

%

Among exp day

%

Among lab

%8 24 7.48 4% 11% 9% 76% 0.3252 0.6669

6543210

1.6

1.4

1.2

1.0

0.8

0.6

0.4

0.2

0.0

Mean LR

Re

pe

atab

ility

SD


Dis.

Repeatability SD vs lab means

6543210

1.8

1.6

1.4

1.2

1.0

0.8

0.6

0.4

0.2

Mean LR

Re

pro

du

cib

ility

SD


Dis.

Reproducbility SD versus lab means

12.0 Precision and Bias 12.1 Precision:12.1.1 An interlaboratory study (ASTM ILS #650) of this test method was conducted at eight laboratories testing three disinfectants (non-chlorine oxidizer, phenol and quaternary ammonium compound) at 8 concentrations (depicted in Fig. 2). An ANOVA model was fit with random effects to determine the resemblance of the untreated control data and the repeatability and reproducibility of the treated data. 12.1.2 The reproducibility standard deviation was 0.67 for the mean log densities of the control biofilm bacteria for this protocol, based on averaging across eight wells on each plate. The sources of variability for the untreated control data are provided in Table 1.

Table 1. Untreated control data variance assessment.

12.1.3 The repeatability (Fig. 5) and reproducibility (Fig. 6) of each disinfectant at each concentration is summarized.

12.1.4 For each of the three disinfectant types considered, the protocol was statistically significantly responsive to the increasing efficacy levels. The log reduction of the non-chlorine oxidizer increased by 0.87 for each increase in efficacy level. The log reduction of the phenol disinfectant increased by 0.87 for each increase in efficacy level. The log reduction of the quat increased by 0.5 for each increase in efficacy level.

12.2 Bias:12.2.1 Randomization is used whenever possible to reduce the potential for systematic bias.

Summary

Even though biofilms are complicated, it is feasible to develop biofilm methods that meet the “Seven R” criteria

Good experiments use control data and randomization.

Invest effort in more experiments versus more replicates (coupons or wells) within an experiment.

Assess uncertainty by SEs and CIs.

For additional statistical resources for biofilm methods, check out: http://www.biofilm.montana.edu/category/documents/ksa-sm

Center for Biofilm Engineering

A National Science Foundation Engineering Research Center established in 1990

www.biofilm.montana.edu

Questions?

Documents

Center for Biofilm Engineering Al Parker, PhD, Biostatistician Center for Biofilm Engineering Montana State University Statistics and Biofilms June 29,