Embed Size (px)
Citation preview
Getting the statistics right for integrative research involving
Ayurveda
Ashwini Mathur(Novartis Healthcare Pvt. Ltd, Hyderabad)
August 1, 2013, Samyukti 2013
1
Disclaimer/Acknowledgements
• All views expressed are authors’ and do not reflect the views of Novartis.
• Images taken from the internet are freely available and are not copyright protected.
• Vinay Mahajan (Novartis)• Vivek Sanker, Sriranjini Jaideep, Ashwini VK (Institute
of Ayurveda and Integrative Medicine, Bengaluru)• Girish Tillu (Symbiosis School of Biomedical Sciences,
Pune)
2
GOOD SCIENCE – Ethical, scientifically unbiased,
transparent
FRAUDOpaque, Secretive
X
LUCKOpaque, Secretive“BAD” SCIENCE
BIASED“BAD” SCIENCE
DECISIONSWHAT DECISION WAS TAKEN ?
CORRECT INCORRECT
HO
W D
ECISION
WAS TAKEN
?
KNOWINGLY
UN-KNOWINGLY
INTRODUCTION
Apr 19, 2012 Statistical Designs in Clinical Trials 3
Background
• Scientific Questions is the “driver”– Statistics as a scientific endeavor is one of the
tools that can help answer the question• Statistical science is the “driver”
– Statistics as a scientific endeavor is one of the tools that can help generate the question that needs to be answered.
4
Background
• Basic issues related to statistical science that can lead to biased results– Design of trial– Sample size– Multiple Decisions from one experiment
5
Example
• Team visits IHST Campus to evaluate the research facilities available for conducting Ayurvedic clinical research– Aim is to evaluate academic credentials of staff– Infrastructure to conduct Ayurvedic treatments– Quality control of Ayurvedic formulations
• While visiting IHST, a team of 4 evaluators, carry out their research and prepare their report
• One of the evaluators finds a Rs. 1000 note at the entrance of IHST
• What do you do?
6
Example
• Publish report saying:-– IHST is suitable for clinical research and
• Everyone who goes their will find Rs. 1000• Rs 1000 was found but is in-consequential• Rs 1000 was found and it will be found again and
reason is that it is an area of high people mobility• Rs 1000 was found but we cannot say anything as this
experiment was not designed to look for Rs 1000.• To prove that Rs. 1000 is a real finding, a designed
experiment needs to be done.– How ?
7
• Respect, Beneficence, Justice.
Ethics - Bedrock of all clinical research.
• Be factual about planning, conduct and reporting• Ensures that each step is taken after due thought process • Creates confidence in the consumers of the research endeavor.
Transparency
• Based on scientific and communication excellence. • Interpretation is based on internal validity and generalizability. • Communication should ensure that clinical researcher and consumers of this
research are closer to the truth. • Note that a particular research endeavor by design could be biased but with
proper communication (for e.g. by acknowledging this bias) it could result in coming closer to the truth and as such becoming scientifically unbiased.
Scientific Un-biasedness
Apr 19, 2012 Statistical Designs in Clinical Trials 8
CLINICAL RESEARCH ....
Statistics • Lies, Damned Lies, Statistics – Mark Twain
• Lies, Damned Lies and THEREFORE Statistics
• Correct Answer to Incorrect Question
• Approximate Answer to Correct Question
9
• hypotheses testing (scientific unbiasedness and transparency)• confidence intervals (scientific unbiasedness)• sample size (scientific unbiasedness and ethics)• randomization and blinding (scientific unbiasedness and transparency)• bias and bias reduction (scientific unbiasedness and transparency)• statistical interpretation of data (scientific unbiasedness and transparency)• meta-analyses (scientific unbiasedness)• design of experiments – ethics, scientific unbiasedness, transparency
Statistical principles can play a role in ensuring that clinical research is aligned along these principles.
Apr 19, 2012 Statistical Designs in Clinical Trials 10
CLINICAL RESEARCH AND STATISTICAL PRINCIPLES
General Problems in Clinical Trials Publication
• Lack of transparency • Publication bias• Scientific issues with trial design • Ethical issues in reporting• Publications generate many more questions
than provide answers!
11
Methodology, Quality and Scope• Transparency issues
– Trials published in many journals not indexed by the databases
– Trials conducted could have been reported in language other than English.
– Tendency of publishing more positive studies vs. negative studies. This publication bias would result in building biased scientific literature.
• Scientific issues– Studies were of short duration with lesser patients. The duration may not
reflective of the true clinical setting, giving rise to meaningless results. Smaller studies tend to overestimate the treatment effects.
– Methodological quality of the trials was suboptimal.
– Randomization, single arm studies
• Ethical issues– Smaller sample sizes
– Negative unpublished work
– Application of western endpoints to traditional methods served
12
Statistical Issues
• Design and sample size– Address issues related to
• Complex Interventions• Unknown effect size
• Analysis– Address issues related to bias due to design
13
Sample Size
• Sample Size– Depends on effect size, “noise”, Error
• Sample Size if not specified– Unethical– Large study or Small study– Interpretation of results is problematic
• Biological significance vs. statistical significance• Negative results – due to ineffective treatment or due
to more than planned “noise”
14
Sample Size
• Sample size could be estimated to do Hypothesis testing provided a clear hypothesis is stated
• Sample size could also be calculated to “estimate” the effect size
15
Design
16
• Whole system Ayurvedic Interventions are complex– multiple component intervention and adjustment of the components
depending on the individual
• Western biomedicine to a large extent, the interventions have been “simple” which have allowed double blind randomized clinical trials.
• Many situations, even in the western biomedicine where these “ideal” trials are infeasible and in these cases non randomized un-blinded trials, observational studies, case studies and case series have been used.
Design
17
• Some examples– Evaluation of public health interventions– Trials in therapeutic areas such as oncology and psychiatry– Medical device trials– Trials which involve invasive interventions like surgery.
• Trials in these areas have biases associated with them and a goal for these trials is as much about understanding the intervention as it is about understanding the limitations and biases associated with the trial itself.
Design
18
• In my opinion it is much better to get an approximate answer (biased results) to the exact question (for e.g whole system intervention for aging as a multi-center observational study at Ayurveda Hospitals) compared to an exact answer (unbiased results) to an approximate question (simplified intervention, for e.g. only using a capsule made of the traditional herbs and doing a multi-center double blind randomized study).
• Evidence from non-randomized designs is more convincing when – confounders are well-understood and measured– there is historical evidence which has a theoretical basis– effect sizes are large.
Ayurvedic interventions lend them into this category where non randomized trials should be okay.
Design
19
• Groups such as all patients in a particular Ayurveda center are randomly allocated to the whole system intervention or a control intervention
• Care should be taken that all characteristics of the centers – for e.g. size of the center, number of vaidyas in each center, experience of staff, research center or healthcare center are balanced.
• Randomization of the centers should take care of balancing the above mentioned characteristics
Cluster Trials
• Stratification can be used to balance characteristics during planning case.• In case characteristics do not match, statistical methods like regression or
propensity scores could be used.
Notes on the design
Design
20
• If patients have very strong preferences among treatments, basing treatment allocation on patients’ preferences may be appropriate.
• In Ayurveda trials, a subjects’ preference for Ayurveda intervention could be very high or low. In this case allocating Ayurveda intervention to a subject with high preference for Ayurveda might be appropriate.
Preference Trials and Randomized consent designs
• Note that if the above policy of allocation is used, the trial is non-randomized.• Comprehensive Cohort Designs, Two Stage Designs, Randomized consent designs
are various modifications of adjusting the design according to the subject’s preference and could be used
Notes on the design
Design
21
• Subjects are measured at baseline and then again after the intervention. • Subjects can receive a control or experimental treatment, but the rule for
assignment to (selection into) treatment conditions is unknown to the researchers.• Important tool for determining the effectiveness of an intervention in routine
clinical practice. • Such trials can have cohort or case-control designs
Observational studies (also known as nonequivalent-control-group design)
• Due to nonrandom nature or non-equivalent nature of these trials with respect to the treatment groups, participants in the two groups may have different histories, or baseline and outcome measures.
• Confounders must be measured carefully and correctly to minimize the biases arising due to selection of subjects or due to performance bias due to non-blinded nature
Notes on the design
Design
22
• measure performance before and after the introduction of an intervention in the same individual and any observed differences in performance are assumed to be due to the intervention
• An extension of this simple design is the interrupted time series (ITS) designs where multiple measures before and after the intervention are made. In contrast to simple before and after designs, ITS designs allow for assessing intervention effects as compared to underlying time trends that might coincide with the before and after measurements
Before and After Clinical Trials
• Addition of a control group would make the before and after trial even more robust, so for e.g. before and after time series measurements in a control and intervention group would be a good robust design in-lieu of a randomized clinical tril.
Notes on the design
Analyses
• Propensity scores method is useful where many confounders need to be controlled for but the data are limited.
• The principle is based on the fact that propensity scores capture the information about the relationship between confounders and treatment allocation (not the outcomes as is the case in stratification and regression techniques), so that selection bias is removed when comparisons are made between groups with similar propensity scores.
• In many Ayurveda Trials, selection bias could be a major component of the overall bias due to non-randomized nature of allocating the interventions
23
Analyses
• If confounding variables or characteristics which determine the allocation are captured correctly, then the bias associated with the selection bias could be removed using the propensity score method
• The method involves calculation for each subject their chance of receiving the experimental intervention from their baseline characteristics or in other words estimates a subject’s propensity of receiving the experimental intervention based on his or her characteristic
24
Analyses
• In a randomized trial with two equal sized treatment groups, the propensity will be the 0.5 for each subject and will not depend on his or her characteristic.
• In non-randomized trials, for example for an Ayurveda intervention where two treatment groups are Ayurveda whole system intervention and normal western biomedicine intervention, it is likely that treatment assignment depend on baseline characteristics.
25
Analyses
• It might be that patients with diagnoses of the disease which is closer to how it is described in traditional Ayurveda texts may be more likely to receive Ayurveda intervention.
• In this case the average propensity score in the Ayurveda intervention group will differ from the average in the western biomedicine group. In this case selection bias is a problem that needs to be addressed and propensity score method can be used to do that.
26
Discussion
• RCT is the preferred method of assessing intervention effects but not at the cost of diluting the intended intervention
• Statistical methods exist which allow for designing and analyzing pragmatic whole system trials
• Guidelines for designing, analyzing and reporting Ayurvedic whole system trials need to be developed.
27
28
Development of Reporting Standards
Guidelines should have 3 basic attributes such that the reporting reflects · Transparency: reporting must be honest and
accurate· Science: reporting must be scientifically unbiased· Ethics: reporting should be based on trials which
are ethically conducted
