Meandering journey towards routine trial adaptation: Survey …/file/... · 2015-06-11 · Meandering journey towards routine trial adaptation: Survey results on the use of adaptive

Meandering journey towards routine trial adaptation: Survey results on the use of adaptive designs in confirmatory trials

Meandering journey towards routine trial adaptation: Surveyresults on the use of adaptive designs in confirmatory trials

Munya Dimairo*, Susan Todd, Steven Julious and Jon Nicholl

Clinical Trials Research Unit, University of Sheffield

[email protected]

12 May 2015PSI Annual Conference (2015)

London


Outline

1 Some background information

2 Addressing the gap

3 The findings

4 Discussion and conclusions

5 Acknowledgements

6 References


Some background information

Introduction

The concept and methodological developments of ADs have come of age

... there can be no objection to the use of data, however meagre, as a guide to action required

before more can be collected ... Thompson,1933

1 However, practical implementation of AD has been too slow to gain traction

2 Understanding cross-disciplinary barriers and concerns is paramount

3 Some previous research (Quinlan et al,2010; Morgan et al,2012; Kairalla et al,2012;

Jaki,2013)

Lack of systematic approach to inform themes

Much focus on early phase trials

Setting

Little cross-sector and cross-disciplinary approach


Some background information

Aims

1 Explore barriers and concerns, and rank them with respect to perceived importance for

prioritisation

2 Explore ways to address some perceived barriers and propose recommendations

3 Comparison of private and public sector perspectives

4 Explore types of confirmatory ADs being implemented in practice


Addressing the gap

Survey approach

Uncovered some concerns which we could have overlooked


Addressing the gap

Survey instrument


Addressing the gap

Outline of statistical analysis

Descriptive

Rasch analysis using a Rating Scale Model (Andrich,1978)

ln(pnik

pni(k−1)

) = θn − (δi + τk ) (1)

k = 0, 1, ...,m are ordered responses and m is the number of choices

pnik - probability of respondent n with ability θn choose score k for item i

δi - perceived importance of item i

τk - threshold parameter corresponding to choice k

θn and τk are nuisance parameters


The findings

Snapshot of underlying distribution of the perception of barriers

Barrier item Not an Somewhat an Moderately Extremely

important important important important

barrier barrier barrier barrier

Lack of practical knowledge 1(8%) 3(23%) 5(38%) 4(31%)

Limited time to support planning 1(8%) 6(46%) 2(15%) 4(31%)

Practical complexities 1(8%) 3(23%) 6(46%) 3(23%)

Data management infrastructure 3(23%) 5(38%) 2(15%) 3(23%)

Lack of applied training 2(15%) 2(15%) 7(54%) 2(15%)

Lack of hands-on experience 1(8%) 4(31%) 6(46%) 2(15%)

Insufficient access to case studies 2(15%) 5(38%) 4(31%) 2(15%)

Preference for traditional designs 3(23%) 2(15%) 5(38%) 3(23%)

Lack of awareness of acceptable scope 1(8%) 4(31%) 6(46%) 2(15%)

Amount of time & effort to support planning 4(31%) 4(31%) 2(15%) 3(23%)

Regulatory fear 5(38%) 1(8%) 3(23%) 3(23%)

Marketing challenges to key stakeholders 2(15%) 6(46%) 4(31%) 1(8%)

. . . . .

. . . . .

. . . . .

Lack of motivational R & D support 6(46%) 5(38%) 2(15%) -

Negative regulatory experiences 5(38%) 3(23%) 2(15%) -

Lack of planning expertise 5(38%) 4(31%) 4(31%) -

Costing complexities 6(46%) 3(23%) 2(15%) 1(8%)

Negative implementation experiences 7(54%) 2(15%) 2(15%) -

Lack of R & D financial support 8(62%) 2(15%) 2(15%) 1(8%)

Lack of statistical expertise 8(62%) 2(15%) 3(23%) -

Worry about employment contracts 10(77%) 2(15%) - -


The findings

Private sector’s perception of barriers

Lack of practical knowledgeLimited time to support planning

Practical complexitiesData management infrastructure

Lack of applied trainingLack of practical hand-on experience

Insufficient access to case studies

Lack of awareness of acceptable scopeAmount of work and time required

Regulatory fearMarketing challenges to key stakeholders

Lack of awareness of when appropriateChallenges setting up decision-making criteria

Stats design complexitiesLack of bridge funding

Qutsourcing challengesTension in decision-making

Stats implementation complexitiesLack of awareness of benefits

Lack of awareness of implementation resourcesLack of knowledge of stats softwareLack of motivational R & D support

Negative regulatory experiencesLack of planning expertise

Costing complexitiesNegative implementation experiences

Lack of R & D financial supportLack of statistical expertise

Worry about employment contracts

Preference for traditional designs

-2 0 2 4 6Perceived relative importance


The findings

Lack of practical knowledge and experience

1 Learning from pacesetters - few organisations frequently implementing ADs

2 Encourage more accessible publication of case studies: rationale, design, regulatory hurdles,

statistical and practical aspects, implementation resources, facilitators, etc

3 Better indexing of ADs related publications

4 Transparency and optimal reporting of ADs in publications

ADs tailored CONSORT statement?

Practical complexities

1 Logistics of how the design will work in practice?

2 Degree varies considerably depending on the nature of the AD

Better communication of the complexity message

Practical oriented toolkit of design specific issues researchers need to consider when

planning confirmatory ADs


The findings

Fear of risking regulatory approval

1 Regulatory engagement: advice consultations

2 Adequate description of;

Rationale, type and scope of proposed AD

Procedures to minimise operational bias

Methodological issues (type I error rate control, inference after adaptation, potential

population drift, etc)

Totality of confirmatory evidence

Sponsor involvement (cross-sector of acceptable models needed)

Who is involved in interim decision making?

Consideration for important secondary objectives (e.g safety)

3 Adequate documentation of processes and procedures

4 Familiarisation with regulatory guidance and reflection papers

5 Improving regulatory awareness and experiences (Elsaßer et al., 2014)


The findings

Difficulties in selling ADs to key stakeholders

1 Better communication of the rationale of AD is imperative

2 How the design addresses the research question(s)?

3 Illustration of potential tangible benefits (patients, trial duration, resource utilisation, etc)

Lack of awareness acceptable scope and when appropriate

1 Cross-sector/disciplinary guidance document?


The findings

Contrast with public sector perspective

Lack of bridge fundingLack of practical knowledge

Lack of practical hand-on experience

Marketing challenges to key stakeholdersAmount of work and time required

Limited time to support planningLack of applied training

Insufficient access to case studiesPractical complexities

Stats design complexitiesChallenges setting up decision-making criteria

Capacity of proposal developersCosting complexities

Lack of awareness of acceptable scopeLack of awareness of when appropriate

Worry about employment contractsLack of awareness of implementation resources

Regulatory fearData management infrastructure

Stats implementation complexitiesLack of knowledge of stats software

Lack of statistical expertiseLack of awareness of benefits

Tension in decision-makingNegative implementation experiences

Negative reviewers experiences

Preference for traditional designs

-2 -1 0 1 2 3Perceived relative importance

Perception of barriers is consistent across sector

Few exceptions reflecting differences in organisational structures, experiences and nature of

study interventions


The findings

Private sector perceptions of concerns

Incr

easi

ng d

egre

e of

con

cern

Impact on secondary objectives

Potential of operational bias

Population drift

Robustness in decision-making

Acceptability to change practice

Non-inferiority early stopping

Efficacy early stopping

Futility early stopping

-1 -.5 0 .5 1 1.5Perceived relative importance

Small degree of separation in perception

More receptiveness towards futility early stopping


Discussion and conclusions

Implications of the findings

1 There are still multifaceted barriers requiring addressing

2 Cross-sector and cross-disciplinary engagement is key to alleviating some of the challenges

3 Most barriers emanate from the lack of practical knowledge and experience

4 Cross-sector/disciplinary receptiveness towards futility early stopping

Limitations

1 Average response rates

2 Sample representativeness

3 Findings may provide conservative perceptions on some of the barriers and concerns


Acknowledgements

Acknowledgements

1 Funding: NIHR Doctoral Research Fellowship (DRF-2012-05-182)

2 Fellowship Supervisors

Prof Steven Julious

Prof Susan Todd

Prof Jon Nicholl

3 Personal Tutor: Mike Bradburn

4 Advisory panel members

5 Participating organisations and individuals


References

References

Andrich, D. (1978) A rating formulation for ordered response categories. Psychometrika. 43

(4), 561–573

Elsaßer, A. et al. (2014) Adaptive clinical trial designs for European marketing authorization:

a survey of scientific advice letters from the European Medicines Agency. Trials. 15 (1), 383.

Jaki, T. (2013) Uptake of novel statistical methods for early-phase clinical studies in the UK

public sector. Clinical trials. 10 (2), 344–346

Kairalla, J. a et al. (2012) Adaptive trial designs: a review of barriers and opportunities.

Trials. 13145.

Morgan, C. C. et al. (2014) Adaptive Design: Results of 2012 Survey on Perception and Use

. Therapeutic Innovation & Regulatory Science . 48 (4 ), 473–481.

Quinlan, J. et al. (2010) Barriers and opportunities for implementation of adaptive designs

in pharmaceutical product development. Clinical trials. 7 (2), 167–173.

Thompson, W. R. (1933) On the Likelihood that One Unknown Probability Exceeds Another

in View of the Evidence of Two Samples. Biometrika. 25 (3/4), 285–294

Documents

Meandering journey towards routine trial adaptation: Survey …/file/... · 2015-06-11 · Meandering journey towards routine trial adaptation: Survey results on the use of adaptive