Solas Webinar Slides

Statistical Solutions | Registered in Ireland, Reg. No 233638 | www.statsols.com

Visual Sensitivity Analysis

for Missing DataAodhán O’LearyAndrew Grannell

Webinar4th June 2015


Overview• Missing Data Analysis & Imputation• Sensitivity Analysis & Tipping Points• SOLAS Tipping Points Demo• Round-up and Q&A


Aodhán O’Leary

Statistician

MSc. (Stat.) , University College Cork

Andrew Grannell

Head of Research and Development

MSc. (Stat.), University College Cork

Who we are


Missing Data Analysis

• Missing Data Mechanisms (MCAR, MAR, MNAR)

• Complete Case analysis…• LOCF, LVCF, BOCF, BVCF…• Inverse probability weighting• Maximum Likelihood estimation• Multiple Imputation• And others…


Missing Data Analysis

• Missing Data Mechanisms (MCAR, MAR, MNAR)

• Complete Case analysis…• LOCF, LVCF, BOCF, BVCF…• Inverse probability weighting• Maximum Likelihood estimation• Multiple Imputation• And others…


Imputation Overview

• “We use MI to replace missing values with plausible values in order to exploit the information that was recorded”

- Andridge & Little, 2010

• MI methods can be classed into two main types– Model based methods– Hot deck methods

• Techniques incorporate variability to give a number of imputed values for each missing data point


Sensitivity Analysis

• Consists of THREE main steps:1. Drawing conclusions under working

assumptions about missing data,2. Identifying a set of plausible alternative

assumptions,3. Studying the variation in the statistical

output & conclusions under the alternative settings


Regulatory Advice on Sensitivity Analysis

• “Examining sensitivity to the assumptions about the missing data mechanism should be a mandatory component of reporting.” [1]

• “A targeted range of…‘sensitivity analyses’ can help to investigate and understand the robustness of estimates…” [2]

• “‘Tipping point’ analysis recommended by NRC appealing for regulators” [3]


Regulatory Advice on Sensitivity Analysis

• “This…survey of Japan- and foreign-based pharmaceutical manufacturers revealed that missing data are frequent in confirmatory trials…the most popular method for handling missing data was LOCF imputation…LOCF is not justified in disease areas in which missing data are frequent” [4]

• “One type of supportive analysis might investigate treatment effects according to a range of different estimands.” [1]


Treatment Control

Treatment Effect Estimation


Treatment Control

Treatment Effect Estimation¿𝑆𝑢𝑐𝑐𝑒𝑠𝑠

¿𝐹𝑎𝑖𝑙𝑢𝑟𝑒


Treatment Control

Treatment Effect EstimationTreatment: Control:

¿𝑆𝑢𝑐𝑐𝑒𝑠𝑠



Treatment Control

Treatment Effect EstimationTreatment: Success: Control: Failure:




Treatment Control




𝜏=(∑𝑖=1

𝑁

𝑦 𝑖(1)−∑𝑖=0

𝑁

𝑦 𝑖(0)) /𝑁


Treatment Control




𝜏=(∑𝑖=1

𝑁

𝑦 𝑖(1)−∑𝑖=0

𝑁

𝑦 𝑖(0)) /𝑁 �̂�=𝑦𝑇− 𝑦𝐶


T C T C

Effect Estimation with Missing Data


T C T C


�̂�=𝑦𝑇− 𝑦𝐶


T C T C


�̂�=𝑦𝑇− 𝑦𝐶 ~𝜏= 𝑦𝑇𝑁 𝑇+𝑦𝑇𝑁 𝑇

𝑁𝑇 −𝑦𝐶𝑁𝐶+𝑦𝐶𝑁𝐶

𝑁𝐶 obs obs obs obs mis mis mis mis


Tipping Point Plotting



¿

¿

Control Group

Treatment Group

Worst Case



¿

¿

Control Group

Treatment Group

¿

¿

Worst Case Best Case

Control Group

Treatment Group


Enhanced TP Plots Background

• Uses the above plots as the basis for analysing missing data.

• Includes the data from imputations.• Makes it possible to compare the results

of analysis under multiple assumptions.• Gives a more detailed visual analysis,

and allows you to compare effect estimates.


Constructing Enhanced TP Plots

1. Get # of missing outcomes for Treatment & Control group (outline x and y axis range),

Set Up the Axes



2. Potential # of success in Treatment group on x-axis and potential # of success in Control on y-axis,


Set Up the Axes




3. Calculate estimated treatment effect () and p-values for a given hypothesis for each cell,


Set Up the Axes





4. Set up grid to highlight all combinations that result in rejecting the null hypothesis,


Draw the Tipping Point ‘Staircase’





4. Set up grid to highlight all combinations that result in rejecting the null hypothesis,

5. Overlay imputation information (squares, contours)


Overlay Imputation Info


Enhanced Tipping Point Plotting



Overview of SOLAS

• Stand alone Missing data package• FIVE Multiple Imputation techniques• Drag & Drop interface• SOLAS syntax language feature• Plotting & Visualisation techniques


ETP Plots - 1000 Imputations












SOLAS Tipping Points Demo


Round Up

• Background & History

• Overview of Missing Data analysis, Imputation and Tipping Points

• SOLAS Demo


ICH Expert Group Survey

• The purpose of this survey is to gather data on related current practices, primarily focused on clinical trials involving drugs, vaccines and biologics. The survey is being distributed to various regions within ICH, including industry and regulatory communities. If a question is not applicable to your job role, please select “I am unable to answer this question/not applicable to my job role”.

• The ICH E9 Revision 1 working group thanks you in advance for your participation. Please complete the survey by Friday 5th June 2015.

• Link to survey: https://www.surveymonkey.com/r/VPMV57Z


Questions?

Thank you

[email protected]

ICH Survey: https://www.surveymonkey.com/r/VPMV57Z


References

1. NRC-Panel (2010) The Prevention and Treatment of Missing Data in Clinical Trials. National Academies Press.

2. ICH E9(R1) (2014) Addendum to Statistical Principles of Clinical Trials. CPMP/ICH/363/96.

3. LaVange, L.M. (February 2015) Missing Data Issues in Regulatory Clinical Trials. Symposium conducted at the meeting of the Japanese Pharmaceutical Manufacturers Association.


References

4. Tanaka, S., Fukinbara, S., Tsuchiya, S., Suganami, H., and Ito, Y.M. (2014). Current Practices in Japan for the Prevention and Treatment of Missing Data in Confirmatory Clinical Trials: A Survey of Japanese and Foreign Pharmaceutical Manufacturers.Therapeutic Innovation & Regulatory Science, 48(6) 717-723.

5. Campbell, G., Pennello, G., and Yue, L. (2011). Missing Data in the Regulation of Medical Devices. Journal of Biopharmaceutical Statistics, 21(2):180–195.

6. Victoria Liublinska and Donald B. Rubin (2014). Sensitivity analysis for a partially missing binary outcome in a two-arm randomized clinical trial. Statistics in Medicine, 33(24):4170-4185.

Software

Solas Webinar Slides