Issues in Treatment Study Design

John Whyte, MD, PhDNeuro-Cognitive Rehabilitation Research

NetworkMoss Rehabilitation Research Institute

Topics to Address

• Developmental trajectory of rehabilitation treatment research (analogous to pharmaceutical phases)

• Treatment theory• Implications for subject selection, choice of

outcome measures, overall study design

Treatment Development

Pharmaceutical Model

• Treatment is well defined, only dosage is in question

• Phase I: Primarily safety, dose finding• Phase II: Small, often single center studies to

establish preliminary evidence of efficacy• Phase III: Large, multicenter test of

efficacy/effectiveness against primary outcome• (Phase IV): Post-marketing surveillance of

actual use

Rehabilitation Model

• Formalize a treatment protocol that is repeatable*• Assess safety of the treatment*• Establish a treatment effect• Establish parameters for optimal “dose” and

schedule of treatment• Examine probable mechanisms of treatment

effect*• Identify or develop outcome measures that

capture both change and impact*

Rehab Model (cont.)

• Identify the treatment’s efficacy in a population and in subgroups of that population

• Determine the treatment’s generalizability• Determine the impact of the treatment on

broader, real-world outcomes• Determine the effectiveness of the treatment as

delivered in real-world healthcare delivery systems to diverse patients§

• Assess cost effectiveness of the treatment

Treatment Theory

• A proposed specification of the “active ingredients” of treatment

• A proposed specification of the mechanism of action of those active ingredients

Framing Treatment Theory Related to Electrical Stimulation

• Acceleration of natural recovery mechanisms• Direct reorganization of specific neural networks• Alteration of the balance among competing

networks• Temporary facilitation/inhibition that alters

learning potential (i.e., must be coupled with experience)

• General theory applicable to heterogeneous neural systems, or specific theories related to specific neural/functional domains?

Treatment Theory Informs Other Aspects of Study Design

• Selection of appropriate study participants• Selection of appropriate outcome

measures• Selection of the optimal experimental

design

Study Participants

• Given the proposed mechanism of action, who can realistically respond? (early phase – rehabilitation research)– Presence of intact homologous system?– Presence of some residual capacity in injured

system (assessed neurophysiologically or behaviorally)?

– Ability to engage in experiential learning?

Study Participants (cont.)

• Of those able to respond to the treatment mechanistically, who will show meaningful functional gains (later phase, enablement research)?– Functional domains likely to show improvement in

principle– Relevant coexisting impairments– Selection of individuals with “pure” impairments,

measurement of confounding impairments, or multifaceted treatments?

Characterizing the Participants

• Once selected, one must ask whether the participants in different treatment groups are “comparable” – comparable in what way(s)?– Balance achieved between randomized groups– Statistical adjustment of differences in

observational studies– Need prior data on prognostic factors

Comparability & Adjustment

• Comparability and adjustment in terms of prognosis on the outcome measures chosen

• Comparability and adjustment in terms of the characteristics that predict responsiveness to the treatment under study

Outcome Measures

• Given the proposed mechanism of action of the treatment, where would you expect to see treatment impact?– The most “proximal” impact (as evidence for the

proposed mechanism)– A more “distal” impact (as evidence that the

change achieved has clinical meaning, ecological validity)

• What other factors are likely to modify the chosen outcomes?

Outcome Measures (cont.)• The same treatment may be judged effective

or ineffective, depending on the outcomes chosen:– Oral antispasticity medications– Memory remediation treatments

Overall Study Design

• Does the proposed mechanism of action suggest a permanent effect of intervention? (parallel group vs. crossover design)– What about “temporary facilitation” of “long-term

learning”?

• Does the proposed mechanism of action suggest “localized” or “generalized” treatment impacts? (feasibility of multiple baseline across behaviors design; utility of multiple outcomes)

Study Design (cont.)

• Does the proposed treatment capitalize on neurologic recovery (early vs. late enrollment)?

• Are there important covariates that might affect treatment response that should be measured?

• Can preliminary evidence of treatment impact be gathered without an untreated control group?

Discussion Issues

General Issues

• What “phase(s)” have been reached in the studies to date?

• Does/can a single investigative group take a treatment from beginning to end, or is there a need to “hand off” or partner with different types of investigators as research progresses?

Treatment Theory

• How have researchers, to date, framed hypotheses about treatment theory, regarding such issues as:– Restitution/compensation?– Passive vs. active (and relationship to experience)?– Duration of action?

• Optimizing treatment prior to large-scale implementation:– Dose/schedule– Generality/specificity– Experiential component

Subject Issues

• In studies to date, have subjects been selected on:– Neuroanatomic and/or behavioral grounds?

(implications for focal vs. diffuse injuries)– Acuity/chronicity?– Status of coexisting impairments?

Outcome Issues

• Optimal ways of confirming basic mechanism, e.g.:– Changes on functional imaging– Changes in threshold for stimulation– Changes in a tightly linked behavioral domain

• (Later) Optimal ways of demonstrating functional impact

Overall Design Issues

• Have studies to date used:– Pre-post treatment designs?– Single subject designs?– Cross-over designs?– Parallel group designs?