Opportunities for Statistical Contributions to Bioassay · Bioassay Fundamentals Changes to USP Bioassay Guidance Changes Driven by Science Equivalence Design Structure Variances

©

D. Lansky

Abstract

BioassayFundamentals

Changes to USPBioassay Guidance

Changes Driven by Science

Equivalence

Design Structure

Variances

Challenges in Bioassay

Summary

Acknowledgements

Opportunities for Statistical Contributionsto Bioassay

David Lansky, Ph.D.

Burlington, Vermont, USA

May, 2012

1 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Abstract

A recent major update to the USP guidance on bioassaypreserved the fundamentals of bioassay statistics. Parts of thisupdate expanded some explanations and examples (i.e., morecomplex design structures), to address issues that have oftenbeen ignored in practice. The update includes conceptualchanges that are important advances for bioassay based oncareful application of statistics; two important changes are:equivalence testing for similarity and guidance on how to deriveassay performance requirements from product specifications.This talk will briefly summarize the practical and statisticalchallenges in bioassays, review the USP guidance, highlightimportant changes then move on to describe recent researchresults that can guide current practice in bioassay as well asindicate where more research is needed. The wrap-up willsummarize areas where there are opportunities for statisticalresearch and improved statistical practice; these are importantopportunities for statisticians to contribute to improved bioassaypractices.

2 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Bioassay Basics

I EC50 varys

I Relative potency iff”biosimilar” active

I ’Biosimilar’ ⇒similar curves

I Reduced modelI potencyI variance

w/Fieller’s

logEC50

log potency

3 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Types of Bioassays

1. Direct (random dose, fixed response)2. Indirect (fixed doses, random responses)

I Discrete response (logit/probit-log)I Quantitative response

I Slope RatioI Parallel Line

I straight lineI nonlinear ⇐I smooth

4 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Assumptions (Parallel Line)

I From a consistent process

I Normal

I Constant Variance

I Independent

I Ref and test have ”biosimilar” activematerial

I Between-assay σ2 of log potency is zero

I Fieller’s works

5 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Violations of Assumptions (1)

I Non-Constant VarianceI TransformI Weight

I Non-normalityI TransformI Change likelihood

I Non-IndependenceI Change assay designI Model design structure

6 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Violations of Assumptions (2)

I Non-Biosimilar:Potency has no meaning

I Non-zero between-assay σ2Log Potency

I Combine potencies w/o wt or ”semi-wt”I Mean log potency & sampling SD

I Fieller’s fails: From 1 assay don’t reportI σ2 of (log) potencyI CI on potency

7 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

USP Big Changes

I Develop, rather than routinely testI Constant varianceI Normality

I Transform recommended vs. wtI Outliers: assess after transform and all-data

minimal-assumptions model

I Assess similarity with equivalenceI Emphasize and explain design structureI Routinely combine potencies w/”simple”I Validation

I Performance requirements from needs (Cp)I Reportable value (geometric mean potency)I Variance components approach

8 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Why Bioassay?

I σ2Log EC50 >> σ2

Log Potency (Blocks)

I Multi-dose test needed (not single-pointw/calibration curve) to assess similarity

I High variance system need replicatesI Parallel line relative potency is:

I Linear: intercept differenceslope

I Nonlinear: difference in Log EC50s

9 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Causes of 6= Variance

I 25 typical reference curvesI Several processes contribute to common

variance patterns:I variation changes with responseI σ2

Log EC50 largeI serial dilution errorI underlying binomial response

Ref curves from 25 assays

log concentration

Response

SD estimated at each concentration

log concentration

SD

(Resp

onse

)

10 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Weight vs. Transform

I Good Weighting Practices:I Weights known (i.e., Probit)I If weights from data

I Non-correlated observationsI Dangers if weights = f (y)

I Good Transform Practices:I residual plotsI theoretical modelI Box-Cox

11 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Setting Equivalence Bounds

I Equivalence (quickly) acceptedI Setting equivalence bounds?

I Assay capability? BAD IDEAI Time-honored: by eyeI Driven by medicine (safe & effective)I Can we do better?

I Parameter-specific boundsI parameters meaningfulI certain combinations particularly badI quality attributes

I Goal: universally meaningful amounts ofnon-similarity?

12 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Universal Non-Similarity

I y ∗ = Ai

1+e−Bi (log(x)−Ci )+ Di + ε

I Responses and parameters in varyingunits

I Scaled Similarity ParametersI %∆A = 100× (ATest − ARef)/ ARef

I %∆D = 100× (DTest − DRef)/ ARef

I %∆B = 100× (BTest − BRef)/ BRef

I Concerns:I Is meaning consistent across assays?I Variances and confidence intervals for

%∆A, %∆B , and %∆D

13 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Interpret ∆A:B :D 10:50:10

% Shifts have consistent meaning

A: Range

B: Slope

D: Lower Asy.

I A and D × (2/3, 1, 2/3)

I B × (1/3, 1, 3)

14 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Factorial %∆A:B :D 5:35:5

Lower Asy -5%

{ -35 }{ -5 }

{ 0 }{ -5 }

{ 35 }{ -5 }

{ -35 }{ 0 }

{ 0 }{ 0 }

{ 35 }{ 0 }

{ -35 }{ 5 }

{ 0 }{ 5 }

{ 35 }{ 5 }

15 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements


Lower Asy 0% change

{ -35 }{ -5 }

{ 0 }{ -5 }

{ 35 }{ -5 }

{ -35 }{ 0 }

{ 0 }{ 0 }

{ 35 }{ 0 }

{ -35 }{ 5 }

{ 0 }{ 5 }

{ 35 }{ 5 }

16 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements


Lower Asy +5%

{ -35 }{ -5 }

{ 0 }{ -5 }

{ 35 }{ -5 }

{ -35 }{ 0 }

{ 0 }{ 0 }

{ 35 }{ 0 }

{ -35 }{ 5 }

{ 0 }{ 5 }

{ 35 }{ 5 }

17 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Remaining Equivalence Challenges

I Some advocate composit testingI Evaluating ”universal” bounds

I 5% asymptote and range seem largeI 35% slope seems okI some combinations likely to induce bias

I Experience with %∆A:B:DI excellent assays can use 5:35:5I noisy assays struggle with 10:50:10

I truncation bias

18 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Quotes from Finney (1978)

A biochemist, pharmacologist, or microbiologist whose own

statistical expertise is small will perhaps object to some of the

designs in later chapters: ...because when he had obtained the

data he would have no idea how to analyze them. This difficulty

illustrates the need for close collaboation between the

experimental scientist and the statistician. ...the right policy is

surely to learn how to analyze the data or to obtain assistance

from a professional statistician.

In framing his advice, the statistician needs to remember that a

simple design can give better results ...

19 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Simple Design?

G

F

E

D

C

B

2 3 4 5 6 7 8 9 10 11

20 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Simple Design?

G

F

E

D

C

B

2 3 4 5 6 7 8 9 10 11

21 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

6-Block Randomized Strip-Plot Design

ABCDEFGH

1 2 3 4 5 6 7 8 9101112

1

1 2 3 4 5 6 7 8 9101112

2

1 2 3 4 5 6 7 8 9101112

ABCDEFGH

3

1 2 3 4 5 6 7 8 9101112

ABCDEFGH

4

1 2 3 4 5 6 7 8 9101112

5

1 2 3 4 5 6 7 8 9101112

ABCDEFGH

6

22 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Mixed Model Fit

Log Concentration

Lo

g R

esp

on

se

1.0

1.5

2.0

2.5

-10 -8 -6 -4 -2 0

11111

1

1111

DDDDDD

DD

DD

HHH

HH

HHHHH

RRRR

R

RRRRR

1

1111

11

1111

DDDDD

DD

DDD

HHH

H

HHHHHH

RRRR

RR

RRRR

2

-10 -8 -6 -4 -2 0

11111

11111

DDDDDD

DDDD

HHHH

HHHHHH

RRRR

RR

RRRR

3

11111

111

11

DDDDDD

DD

DD

HHHH

HHHH

HH

RRRRR

RR

RRR

4

-10 -8 -6 -4 -2 0

1111

1

111

11

DDDDD

D

DD

DD

HHHH

HHHHHH

RRRR

R

RRR

RR

5

1.0

1.5

2.0

2.5

1111

11

1111

DDDDDD

DD

DD

HHH

H

HHHHHH

RRRRR

RRRRR

6

23 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Problems with Variance Estimates

I Without design structure in the model,variance estimates are meaningless

I Experience with real and simulated dataon typical bioassays:

I In linear, Fieller’s substantiallyunderestimates variance of potency

I In nonlinear, model-based variance ofpotency is an underestimate

I Good variance estimators would agreewith sampling variance of log potency

I Sampling variance of log potency fromindependent assays is simple

24 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Combining potencies

I Experience with real (typical) bioassays:I Assays with between-assay variance of log

potency = 0 are very rareI Both from a lab with:

I long and strong experience delivering greatbioassays

I robotsI strip-plot design in assayI strip-plot in analysis

I Conclusion: Don’t expect to weight or”semi”-weight

25 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Highlights of Recent Results

I Avoid Slope Ratio

I Avoid Straight Parallel Line

I Four-parameter logistic broadly robustI Truncation Bias common, to avoid:

I Wide dose range neededI reduce variation around curveI within-assay blocks help

26 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Truncation Bias Simulation

I log 2 target potencies -1.5 to 1.5 (by 0.1)

I CRD

I 4PL data (A=1, B=1.6, C=6.5, D=0)I four doseDesigns:

1 10 doses, 1-12, skip 3 & 102 10 doses 2-113 10 doses 1-12 (= spacing)4 20 doses 1-12 (= spacing)

I residual SD = 0.01, 0.03, 0.05, 0.07

I n = 3, 6, 9

I Similarity criteria = 5:35:5, 10:50:10

I 20 simulated assays at each condition

27 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Potency=0.5, σ = 3%, n=6

0.0

0.2

0.4

0.6

0.8

1.0

2 4 6 8 10 12

R

RR

R

R

R

R

RR R

R R

RR

R

R

R

R

RR

R R R

R

R

R

R

RR R

R RR

R

R

R

R

R R R

RR

R

R

R

R

RR

RR

R RR

R

R

R

RR R

R

T T T T

T

T

T

T

T T

T TT

TT

T

T

T

TT

T T T T

T

T

T

TT T

T TT

T

T

T

T

T

TT

T T TT

T

T

T

TT T

T T TT

T

T

T

T

T T

28 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Truncation Results 1 (Nsim=10)

Similarity Failures

Log 2 Target Potency

Perc

ent Sim

ilarity

Failu

re

0

40

80

-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5

11111111111111111111111111111113333333333333333333333333333333555555

555555

5555555555

5555

5555

5

7777777777777777777777777777777

10:50:10 : n { 3 }

1111111111111111111111111111111

333333333333333333

3333

3333333

33

555555555555555555555555555555577777777777777777777777777777775:35:5 : n { 3 }

111111111111111111111111111111133333333333333333333333333333335555555555555555555555555555

555777777

77777

77777777

77777

77777

77

10:50:10 : n { 6 }

0

40

80

111111111111111111111111111111133

33333333333333333333333333333

55555555555555555555555555555557777777777777777777777777777777

5:35:5 : n { 6 }

0

40

80

1111111111111111111111111111111333333333333333333333333333333355555555555555555555555555555557777777777

777777777777777777777

10:50:10 : n { 9 }

-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5

1111111111111111111111111111111333333

3333333333333333333333333

55555

5555555555555

5555555555555

77777777777777777777777777777775:35:5 : n { 9 }

29 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Truncation Results 2

Similarity Failures 5:35:5


Percent Sim

ilarity Failu

re

0

40

80

-1.5 -0.5 0.5 1.5

1111111111111111111111111111111

3333333333333333333333333333333

55555555555555555555555555555557777777777777777777777777777777 : doseDesign { 1 }

: n { 3 }

1111111111111111111111111111111

333333333333333333333333333333355555555555555555555555555555557777777777777777777777777777777 : doseDesign { 2 }

: n { 3 }

-1.5 -0.5 0.5 1.5

1111111111111111111111111111111

33333333333333333333

33333333333

55555555555555555555555555555557777777777777777777777777777777 : doseDesign { 3 }

: n { 3 }

111111111111111111111111111111133333333333333333333333333

33333

55555555555555555555555555555557777777777777777777777777777777 : doseDesign { 4 }

: n { 3 }1111111111111111111111111111111

3333333333333333333333

333333333

55555555555555555555555555555557777777777777777777777777777777 : doseDesign { 1 }

: n { 6 }

1111111111111111111111111111111

3333333333333333333333

333333333

55555555555555555555555555555557777777777777777777777777777777 : doseDesign { 2 }

: n { 6 }

111111111111111111111111111111133333333333333

33333333333333333

55555555555555555555555555555557777777777777777777777777777777 : doseDesign { 3 }

: n { 6 }

0

40

80

11111111111111111111111111111113333333333333333333333333333333555555

55555555555555555555555

557777777777777777777777777777777

: doseDesign { 4 } : n { 6 }

0

40

80

1111111111111111111111111111111333333333333333

3333333333333333

555555555555555555555

55555555557777777777777777777777777777777

: doseDesign { 1 } : n { 9 }

-1.5 -0.5 0.5 1.5

11111111111111111111111111111113333333333333333333333

333333333

55555555555555555555555555555557777777777777777777777777777777 : doseDesign { 2 }

: n { 9 }

11111111111111111111111111111113333333333333333333333333333333

55555555555555555555555

55555555

7777777777777777777777777777777 : doseDesign { 3 }

: n { 9 }

-1.5 -0.5 0.5 1.5

11111111111111111111111111111113333333333333333333333333333333555555555

55555555555555555555

557777777

77777777777777777777

7777 : doseDesign { 4 }

: n { 9 }

30 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements


PG Bias of Potency 5:35:5


Percent Geometric Bias

-10

0

10

-1.5 -0.5 0.5 1.5

1111111111111111111111111111111333333333

3333333333333333333333

: doseDesign { 1 } : n { 3 }

1111111111111111111111111111111333333333333333333

33333

: doseDesign { 2 } : n { 3 }

-1.5 -0.5 0.5 1.5

1111111111111111111111111111111333333333333

33333333333333333

33

: doseDesign { 3 } : n { 3 }

11111111111111111111111111111113333333333333333333333333333333555555

555555

: doseDesign { 4 } : n { 3 }

111111111111111111111111111111133333333333333333333333333333335555 55

55

555555555

: doseDesign { 1 } : n { 6 }

11111111111111111111111111111113333333333333333333333333333333

55

: doseDesign { 2 } : n { 6 }

1111111111111111111111111111111333333333333333333333333333333355555

5555

555

555555555

: doseDesign { 3 } : n { 6 }

-10

0

10

1111111111111111111111111111111333333333333333333333333333333355555555555555555555555555555557777777 7

: doseDesign { 4 } : n { 6 }

-10

0

10

1111111111111111111111111111111333333333333333333333333333333355555555555555555555555

55555555

: doseDesign { 1 } : n { 9 }

-1.5 -0.5 0.5 1.5

1111111111111111111111111111111333333333333333333333333333333355555555555555555

: doseDesign { 2 } : n { 9 }

111111111111111111111111111111133333333333333333333333333333335555555555555555555555555555555

: doseDesign { 3 } : n { 9 }

-1.5 -0.5 0.5 1.5

111111111111111111111111111111133333333333333333333333333333335555555555555555555555555555555777777777777777777

7777777777777

: doseDesign { 4 } : n { 9 }

31 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements


PGSD of Potency 5:35:5


Perc

ent G

eom

etric

SD

048

12

-1.5 -0.5 0.5 1.5

111111111111111111111111111111133333

333333333333333333

33333333

: doseDesign { 1 } : n { 3 }

1111111111111111111111111111111333333

333333

333

: doseDesign { 2 } : n { 3 }

-1.5 -0.5 0.5 1.5

1111111111111111111111111111111

33

333333333

333333333333333333

33

: doseDesign { 3 } : n { 3 }

1111111111111111111111111111111333333333

33333333333333333333335 5

: doseDesign { 4 } : n { 3 }

11111111111111111111111111111113333333

333333333333333333333333

55 5 5

: doseDesign { 1 } : n { 6 }

111111111111111111111111111111133333333333333333333333333

33333

: doseDesign { 2 } : n { 6 }

111111111111111111111111111111133333333333333333333333333333

3355555

5 5

: doseDesign { 3 } : n { 6 }

04812

1111111111111111111111111111111333333333333333333333333333333355555

555555555555555555555

55555

7 : doseDesign { 4 }

: n { 6 }048

12

111111111111111111111111111111133333333333333333333333333333335555555555555555

5555

5555555

5555

: doseDesign { 1 } : n { 9 }

-1.5 -0.5 0.5 1.5

1111111111111111111111111111111333333333333333333333333333333

3555

5

555 5

: doseDesign { 2 } : n { 9 }

1111111111111111111111111111111333333333333333333333333333333355555555

555555555555555555555

55

: doseDesign { 3 } : n { 9 }

-1.5 -0.5 0.5 1.5

11111111111111111111111111111113333333333333333333333333333333555555

5555555555555555555555555

77777777

77777777

777777

777777

777

: doseDesign { 4 } : n { 9 }

32 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Practical Challenges

I Broad ignorance of design structure

I Bioassay software very limitedI Many constraints on bioassays

I USP/EP guidanceI limited access to statistical supportI 8× 12 platesI complex process, many grouped steps

I Robotics

I many think a fit with smaller residual is(always) better

33 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Statistical Challenges

I Learn, use, and teach design structure

I Get into the lab, work WITH ”clients”

I Robot softwareI Nonlinear RE models on complex design

structures are delicateI sensitive to outliersI must do model selection (HOW?)

I Consider more than bias and SD, failuresmatter

34 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Summary

I Core ideas of bioassay 60+ years old

I Use advances in statistics

I Bioassay uses many statistical methods

I Oopportunities for statisticians inbioassay, people and biology skills matter

I Substantial need for better bioassays

35 / 36

©

D. Lansky

Abstract




Equivalence

Design Structure

Variances


Summary

Acknowledgements

Acknowledgements

I Consulting clients

I USP and USP bioassay panel members

I Stan Deming

I Tim Schofield

I Carrie Wager

I NSF EPSCoR

I NIH SBIR 3R44RR02198-03S1

36 / 36

Documents

Opportunities for Statistical Contributions to Bioassay · Bioassay Fundamentals Changes to USP Bioassay Guidance Changes Driven by Science Equivalence Design Structure Variances