Hypothesis Testing and Adaptive Treatment Strategies

S.A. Murphy

SCT

May 2007

2

Collaborators

• Lacey Gunter

• A. John Rush

• Bibhas Chakraborty

3

Outline

• Adaptive treatment strategies

• Constructing and addressing questions regarding an optimal adaptive treatment strategy

• A solution to non-regularity?

• Example using STAR*D.

4

Adaptive treatment strategies are individually tailored treatments, with treatment type and dosage changing according to patient outcomes. Operationalize clinical practice.

k Stages for one individual

Observation available at jth stage

Action at jth stage (usually a treatment)

5

Goal: Construct decision rules that input information available at each stage and output a recommended decision; these decision rules should lead to a maximal mean Y where Y is a function of

The adaptive treatment strategy is the sequence of two decision rules:

k=2 Stages

6

Data for Constructing the Adaptive Treatment Strategy:

Subject data from sequential, multiple assignment, randomized trials. At each stage subjects are randomized among alternative options.

Aj is a randomized action with known randomization probability.

binary actions with P[Aj=1]=P[Aj=-1]=.5

7

Stages 3 & 4 of STAR*D

Stage 3 Stage 4Preference Treatment Intermediate Treatment

Action Outcome Action

Mirtazapine

Switch R Remission Continue on Current Tx

Nortriptyline

TranylcypromineLithium

AugmentR Non-remission R

ThyroidMirtazapine + Venlafaxine

8

A natural approach: Myopic Decisions

•Evaluate each stage of treatment in isolation; the dependent variable is 1 if remission in that stage, 0 otherwise.

•In stage 3 there are two treatment actions for those who prefer a switch in treatment (Mirtazapine or Nortriptyline) and two treatment actions for those who prefer an augment (Lithium or Thyroid).

•Compare the two switches in treatment according to the remission rate achieved by end of stage 3. Do the same for the two augments.

9

Need an alternative

•This is not a good idea if we want to evaluate the sequence of treatments (e.g. adaptive treatment strategies).

• Some of the stage 3 non-remitters went on to have a remission in stage 4; these people have an dependent variable equal to 0 in the myopic analysis.

•the remission or lack of remission in stage 4 may be partially attributable to the stage 3 treatment.

•Patching together the separate analyses of the stages requires unnecessary causal assumptions.

10

Need an alternative for the stage 3 dependent variable

•What should the value of the stage 3 dependent variable be for those that move to stage 4?

•We should not use a stage 3 dependent variable of Y=1 for those people who remit in stage 4.

•We should not use an stage 3 dependent variable of Y=0 for those people who remit in stage 4.

•The dependent variable should be something in between.

11

Regression-based methods for constructing decision rules

•Q-Learning (Watkins, 1989) (a popular method from computer science)

•A-Learning or optimal nested structural mean model (Murphy, 2003; Robins, 2004)

•The first method is an inefficient version of the second method when each stages’ covariates include the prior stages’ covariates and the actions are coded to have conditional mean zero.

12

There is a regression for each stage.

A Simple Version of Q-Learning –

• Stage 4 regression: Regress Y on to obtain

• Stage 3 regression: Regress on to obtain

13

for patients entering stage 4:

• is the estimated probability of remission in stage 4 as a function of variables that may include or be affected by stage 3 treatment.

• is the estimated probability of remission assuming the “best” treatment is provided at stage 4 (note max in formula).

• will be the dependent variable in the stage 3 regression for patients moving to stage 4

14

A Simple Version of Q-Learning –

• Stage 4 regression, (using Y as dependent variable) yields

• Stage 3 regression, (using as dependent variable) yields

15

Decision Rules:

16

Non-regularity

17

Non-regularity

18

Non-regularity –

• Replace hard-max

• by soft-max

p̂ is the \probability" of selecting a4 = 1.

19

STAR*D Stages 3 & 4

• Regression at stage 3: α3TS3' + β3

TS3A3

•S3' = (1, X3)

•X3 is a vector of variables available at or prior to stage 3

•S3= ((1-Aug), Aug, Aug*Qids2)

• We are interested in the β3 coefficients as these are used to form the decision rule at stage 3.

20

STAR*D Stages 3 & 4

Decision Rule at stage 3:

•If patient prefers a Switch then

•if offer Mirtazapine, otherwise offer Nortriptyline.

•If patient prefers an Augment then

•if offer Lithium, otherwise offer Thyroid Hormone.

21

STAR*D Stages 3 & 4

• Regression at stage 4: α4TS4' + β4S4A4

• S4' =(1,X4, (1-Aug)*A3, Aug*A3, Aug*A3*Qids2),

•(X4 is a vector of variables available at or prior to stage 4, Aug is 1 if patient preference is augment and 0 otherwise)

• S4 = 1

• Decision rule: Choose TCP if , otherwise offer Mirtazapine + Venlafaxine XR

^̄4 >0

22

Switch -.11(.07) -1.6

Augment .47(.25) 1.9

Augment*QIDS2 -.04(.02) -2.3

^̄(s:e:) z statistic

Stage 3 Coefficients

^̄31 =^̄32 =^̄33 =

23

24

Decision Rules from Soft-Max Q-Learning Y=1 if remission or sufficient response to move to follow-up, Y=0 otherwise

Step 3 Step 4 Switch MIRT = NTP (225)

QIDS2 < 11 LI = THY (45) Augment QIDS2 ≥ 11 LI < THY (88)

TCP<VEN+MIRT(104)

= means not significant in two sided test at .05 level

< means significant in two sided test at .05 level

25

Discussion

• We replace the hypothesis test concerning a non-regular parameter, β3 by a hypothesis test concerning a near-by regular parameter.

• These multi-stage regression methods need to be generalized to survival analysis.

• This is work in progress!

26

Discussion

• Robins (2004) proposes several conservative confidence intervals for β3.

• Ideally to decide if the two stage 3 treatments are equivalent, we would evaluate whether the choice of stage 3 treatment influences the mean outcome resulting from the use of the adaptive treatment strategy. We did not do this here.

• Constructing “evidence-based” strategies is of great interest in clinical research and there is much to be done by statisticians.

27

This seminar can be found at:http://www.stat.lsa.umich.edu/~samurphy/

seminars/SCT0507.ppt

Email me with questions or if you would like a copy!

samurphy@umich.edu

28