Upload
medresearch
View
236
Download
2
Tags:
Embed Size (px)
Citation preview
Brian L. Strom, M.D., M.P.H.Chair and Professor, Department of
Biostatistics and Epidemiology
Director, Center for Clinical Epidemiologyand Biostatistics
George S. Pepper Professor of Public Health and Preventive Medicine
Professor of Biostatistics and Epidemiology, Medicine, and Pharmacology
University of PennsylvaniaSchool of Medicine
CCEB
CCEB
“Sampling”
1. What is an appropriate sample size of respondents to best determine risk of sound and look alike proprietary names in the prescription drug study group? In a focus group? In a survey document?
2. Should sample be randomly selected? Is it important to have a statistical significance for this type of evaluation?
CCEB
“Sampling”
• Introduction
• Very general principles of sample size calculations and sampling
• Application of general principles to this situation
• Recommendations for research, to guide the future
CCEB
Options in Research Design
• Analytic Studies– Experimental Study– Prospective Cohort Study– Retrospective Cohort
Study– Case-Control Study
• Descriptive Studies– Analyses of Secular
Trends– Case Series– Case Reports
Case-Control StudiesCase-Control Studies
DiseaseDisease
Coh
ort
Stu
die
sC
oh
ort
Stu
die
s
Fact
or
Fact
or
PresentPresent(cases)(cases)
AbsentAbsent(not exposed)(not exposed)
PresentPresent(exposed)(exposed)
AbsentAbsent(controls)(controls)
AA
DDCC
BB
CCEB
Determinants of Needed
Sample Size• Alpha
• Beta
• Variability (SD)
• Delta--how small a difference do you want to be able to detect?
CCEB
Determinants of NeededSample Size: Cohort
Study• Alpha
• Beta
• Incidence in the unexposed control group
• Delta--how small a RR do you want to be able to detect?
CCEB
Determinants of Needed Sample
Size: Case-Control Study
• Alpha
• Beta
• Prevalence in the undiseased control group
• Delta--how small a RR do you want to be able to detect?
CCEB
Sampling
Study Sample
Conclusion About a Population(Association)
Conclusion About Scientific Theory(Causation)
Statistical Inference
Biological Inference
CCEB
Application of GeneralPrinciples to This
Situation
• THE Central Principleof Research Design: The question is, what is the question??
CCEB
Application of GeneralPrinciples to This
Situation• In this situation, there are no
a priori hypotheses being tested to be able to consider sample size calculations or questions of sampling• What is being performed is
essentially qualitative research
CCEB
Recommendations for Research,
to Guide the Future
**Evaluate the current process in a quantitative fashion**
CCEB
General Strategy for Evaluation
• Standardize procedure• Test for reliability/
reproducibility• Test for validity• Make changes in the
procedure accordingly
CCEB
Standardize Procedure
• Choose among the current possible approaches a “standard” to be evaluated more rigorously
CCEB
Test for Reliability/Reproducibility
• Evaluate the same drug names in the same process with multiple different groups of survey prescribers and experts, to look for whether there is adequate agreement• If no reliability, validity is
impossible and procedure should be abandoned
CCEB
Test for Validity
• Gold standard needed:–Drug names rejected in initial FDA review
–Drug names withdrawn due to problems
–Direct measurement of error rate
CCEB
Retrospective: Drug NamesRejected in Initial FDA
Review
• Problem: was the initial review decision correct?
CCEB
Retrospective: Drug NamesWithdrawn Due to
Problems
• Knowledge of reviewers could be problematic–Other countries–Years ago, and using new
pharmacists
• Was the withdrawal decision a correct one?
CCEB
Prospective: DirectMeasurement of Error Rate
• Simulate real life situation in a study setting
• Choose good and bad options for new names
• Enter possible new names into prescription entry computer program
• Ask large numbers of docs to write orders
• Ask large numbers of pharmacists to “fill” each rx, entering into prescription entry computer program
• Directly measure resulting error rate
CCEB
Make Changes in theProcedure
Accordingly• Determine appropriate cutpoint
for expert ratings, ROC curve vs. gold standard• Determine appropriate sample
sizes through simulation, ie how many are needed to achieve results consistent with the gold standard• Modify processes accordingly
CCEB
Potential Sources of Support
• FDA extramural funds
• AHRQ patient safety funds
• NIA pharmacology program
• NIGMS pharmacology program
CCEB
Conclusions
• Applying a quantitative approach to evaluating what has so far been a qualitative one, could lead to major changes in the procedure, and major improvements in the net results
Statitsically incorrect