Upload
sumberscribd
View
220
Download
0
Embed Size (px)
Citation preview
8/10/2019 Mk Select Right Stat
1/34
Selecting Right StatisticsSelecting Right Statistics
HyungjinHyungjin Myra Kim,Myra Kim, Sc.DSc.D..
The University of MichiganThe University of Michigan
8/10/2019 Mk Select Right Stat
2/34
Choosing an Analytic Method (1)Choosing an Analytic Method (1)
First, analytic plan should be considered whileFirst, analytic plan should be considered while
planning the study.planning the study. What do you plan to study (or measure)?What do you plan to study (or measure)?
Primary outcome measure determines the typePrimary outcome measure determines the type
of dependent variableof dependent variable
Continuous (ex: hours of sleep)Continuous (ex: hours of sleep)
Dichotomous (ex: binge drinking or not)Dichotomous (ex: binge drinking or not) Ordinal (ex: depression diagnosis)Ordinal (ex: depression diagnosis)
Categorical (ex: choice of treatment)Categorical (ex: choice of treatment)
Time to event (ex: time to relapse)Time to event (ex: time to relapse)
8/10/2019 Mk Select Right Stat
3/34
Choosing an Analytic Method (Choosing an Analytic Method (22))
Sometimes, there is no dependent variableSometimes, there is no dependent variable
Factor analysisFactor analysis
Cluster analysisCluster analysis
HigherHigher--way contingency table analysesway contingency table analyses
Agreement (kappa)Agreement (kappa)
CorrelationCorrelation analysis (correlation coefficient)analysis (correlation coefficient)
AAccuracyccuracy ((sensitivity, specificitysensitivity, specificity))
(We will not discuss the above today.)(We will not discuss the above today.)
8/10/2019 Mk Select Right Stat
4/34
Choosing an Analytic Method (Choosing an Analytic Method (33))
Study designStudy design
Do you have a primary comparison?Do you have a primary comparison?
Determines the nature of the primary predictorDetermines the nature of the primary predictorvariable (independent variable)variable (independent variable)
(ex) 2 group or 3 group comparison?(ex) 2 group or 3 group comparison?
(ex) Evaluating the relationship between happiness(ex) Evaluating the relationship between happinessto ratio of leisure to work hoursto ratio of leisure to work hours
How often do you plan to measure?How often do you plan to measure? XX--sectional, longitudinal,sectional, longitudinal,xx--overover
DeterminesDetermines the number of dependent variablesthe number of dependent variables
(ex) pre/post has measurements twice per person(ex) pre/post has measurements twice per person
8/10/2019 Mk Select Right Stat
5/34
The choice of analysis will also depend onThe choice of analysis will also depend on
UnvariateUnvariate vs.vs. bivariatebivariate analysisanalysis
BivariateBivariate vs. multivariate analysisvs. multivariate analysis
Potential confounder?Potential confounder? Adjust for covariates?Adjust for covariates?
Data skewed or sample size small?Data skewed or sample size small? TransformationTransformation
Parametric vs. nonParametric vs. non
--parametriparametri
cc
8/10/2019 Mk Select Right Stat
6/34
Dependent Variable (Outcome)
Study Designs Continuous Binary (yes/no)
Pre/Post Effect of nightly exercise on hrsof sleep before/after in
insomniacs
Patient satisfaction before vs.
after color change in hospital
ward
Matched pairs Mastectomy vs. Lumpectomy onQOL in patients matched by age
& family history
Mastectomy vs. Lumpectomy on
survival in patients matched by
age & family history
1-group Cholesterol in diabetic patients:Is it higher than general public? Depression in substanceabusers
2-group Writing skill between teachingmethods A vs. B
Comparison of drugs A vs. B onrelapse to heavy drinking
2-group,pre/post
Weight before/after in exercisevs. no exercise group
Satisfaction before & afterbetween 2 skin products
3-group Comparing effectiveness of threedrugs on cholesterol
Pain reduction in three
different pain relief medication
ContinuousPredictor
Does pack-year of smoking
predict Cognitive deficit?
Is average nightly sleep
predictive of hair loss?
8/10/2019 Mk Select Right Stat
7/34
What Type of Analysis?What Type of Analysis?
DescriptiveDescriptive
NumericalNumerical tablestables of means, counts,of means, counts,proportionproportion
GraphicalGraphical -- histogramshistograms, box plots, s, box plots, scattercatter
plotsplots,, etc.etc.
InferentialInferential
EstimationEstimation Point estimates/ConfidencePoint estimates/ConfidenceIntervalsIntervals
Hypothesis TestsHypothesis Tests
8/10/2019 Mk Select Right Stat
8/34
Analytic MethodsAnalytic Methods
Dependent Variable (Outcome) Type
Study DesignContinuous
(multiple regression for
multivariate analysis)
Binary (yes/no)
(logistic regression for
multivariate analysis)Pre/Post Paired t-test McNemars test
Matched pairs Paired t-test McNemars test
1-group One-group t-test One proportion test
2-group* Two-group t-test Two proportion test orChi-square Test
2-group, pre/post* Analysis of Covariance or
multiple regression
Repeated measures
logistic regression
3-group* Analysis of Variance Chi-square test
Continuous
Predictor
Simple regression Logistic regression
** BivariateBivariate relationshipsrelationships
8/10/2019 Mk Select Right Stat
9/34
Binary Dependent VariableBinary Dependent Variable
Descriptive Statistics: ProportionDescriptive Statistics: Proportion
To estimate a proportion or prevalence, subjects mustTo estimate a proportion or prevalence, subjects mustbe a representative sample from the population.be a representative sample from the population.
Assuming the subjects are representative andAssuming the subjects are representative andindependent, the rate is estimated as:independent, the rate is estimated as:
p = n/Np = n/N
where n is the number of subjects with the attributewhere n is the number of subjects with the attributeand N is the total number of subjects tested (orand N is the total number of subjects tested (orstudied).studied).
8/10/2019 Mk Select Right Stat
10/34
Binary Dependent Variable (2)Binary Dependent Variable (2)
When Only One Group is of Interest:When Only One Group is of Interest:
TestTest
Proportion compared to a null valueProportion compared to a null value
one proportion testone proportion test
Ex)Ex) Are substance abusers more likely to bedepressed than general public?
Confidence Interval (95% CI: proportionConfidence Interval (95% CI: proportion
1.96*SE)1.96*SE)
Ex) Prevalence of depression in substance abusersEx) Prevalence of depression in substance abusers
Ex) Sensitivity and specificity of a new shortEx) Sensitivity and specificity of a new shortdepression instrument compared with thedepression instrument compared with thephysicianphysicians gold standard depression diagnosiss gold standard depression diagnosis
More on interpretation of 95% CI tomorrow.More on interpretation of 95% CI tomorrow.
8/10/2019 Mk Select Right Stat
11/34
Binary Dependent Variable (3)Binary Dependent Variable (3)
When Comparing Two Independent Groups:When Comparing Two Independent Groups:
Ex) Comparing drugs A vs. B on relapse to heavydrinking
Essentially a 2 by 2 tableEssentially a 2 by 2 table
Comparative TestComparative Test
-- ChiChi--square testsquare test
-- Two proportion testTwo proportion test
Comparative Statistics (summary effect size)Comparative Statistics (summary effect size)
-- Absolute Difference in ProportionsAbsolute Difference in Proportions
-- Odds Ratios (OR)Odds Ratios (OR)
-- Relative Risks (RR)Relative Risks (RR)
For both OR and RR, 1 means no differenceFor both OR and RR, 1 means no difference
Can calculate 95% CI for any of the above (OR, etc.)Can calculate 95% CI for any of the above (OR, etc.)
8/10/2019 Mk Select Right Stat
12/34
Binary Dependent Variable (4)Binary Dependent Variable (4)
When Comparing Two Independent GroupsWhen Comparing Two Independent Groups::
If sample size is smallIf sample size is small
rule of thumb = expected cell count < 5rule of thumb = expected cell count < 5 Comparative Test: FisherComparative Test: Fishers Exact Tests Exact Test
| A B | Total
---------------------------------------------
yes | 3 6 | 9
no | 9 2 | 11
--------------------------------------------Total | 12 8 | 20
Pearson chi-square test p-value = 0.028
Fisher's exact test p-value = 0.065
8/10/2019 Mk Select Right Stat
13/34
Continuous Dependent VariableContinuous Dependent Variable
Descriptive StatisticsDescriptive Statistics
Mean and Standard Deviation if data are symmetricMean and Standard Deviation if data are symmetric
Median and InterMedian and Inter--quartile range if data are skewedquartile range if data are skewed
MeanMean can be affected by one very large or one very smallcan be affected by one very large or one very smallvalue, and therefore isvalue, and therefore is sensitive to outlying valuessensitive to outlying values
MedianMedian isis robust to an outlying valuerobust to an outlying value because it is simplybecause it is simplythe value at the center when data are ranked in orderthe value at the center when data are ranked in order..
IfIf mean and median are very different, data are skewed.mean and median are very different, data are skewed. Always graphically explore the distribution (e.g., usingAlways graphically explore the distribution (e.g., using
histogram, box plot) and choose the appropriatehistogram, box plot) and choose the appropriatedescriptive statisticsdescriptive statistics
More on mean vs. median tomorrow.More on mean vs. median tomorrow.
8/10/2019 Mk Select Right Stat
14/34
Continuous Dependent Variable (2)Continuous Dependent Variable (2)
When Only One Group is of Interest:When Only One Group is of Interest:
Test (One mean compared to a null value)Test (One mean compared to a null value)
One sample tOne sample t--testtest
Ex)Ex) Is cholesterol higher in diabetic patients comparedwith the general public?
Confidence Interval (95% CI = meanConfidence Interval (95% CI = mean
1.96*SE)1.96*SE)
Ex) Sample meanEx) Sample mean cholesterol = 124
Sample SD = 10, N = 200
95% CI for mean cholesterol = 124 1.96*10/sqrt(200)
= (122.6, 125.4)
8/10/2019 Mk Select Right Stat
15/34
Continuous Dependent Variable (3)Continuous Dependent Variable (3)
When Only One Group is of Interest:When Only One Group is of Interest:
When sample size is small (N
8/10/2019 Mk Select Right Stat
16/34
Continuous Dependent Variable (4)Continuous Dependent Variable (4)
When Comparing Two Independent Groups:When Comparing Two Independent Groups:
TestTestTwo independent group tTwo independent group t--testtest
Ex)Ex) Writing skill comparison between teaching methods A vs. B
Comparative Statistics: difference in meansComparative Statistics: difference in means
Ex) Difference in mean writing skill scores betweenEx) Difference in mean writing skill scores betweenthose who were taught with method A vs. method Bthose who were taught with method A vs. method B
Confidence Interval for Difference in MeansConfidence Interval for Difference in Means
95% CI = difference95% CI = difference
1.96*SE (of difference)1.96*SE (of difference)
8/10/2019 Mk Select Right Stat
17/34
Continuous Dependent Variable (5)Continuous Dependent Variable (5)
When Comparing Two Independent Groups:When Comparing Two Independent Groups:
If sample size is small (N < 25) or cannot assume that theIf sample size is small (N < 25) or cannot assume that the
dependent variable is interval and normally distributeddependent variable is interval and normally distributed
Use a NonUse a Non--parametric Test (Test of Median)parametric Test (Test of Median)
WilcoxonWilcoxon ranksumranksum test (tests equality of medians)test (tests equality of medians)
8/10/2019 Mk Select Right Stat
18/34
Graphical Methods to Compare Groups: Box PlotsGraphical Methods to Compare Groups: Box Plots
Resting
Heart
Rate
NoExercise
MildExercise
StrenuousExercise
8/10/2019 Mk Select Right Stat
19/34
Using Subjects as Their Own Controls:Using Subjects as Their Own Controls:
CrossCross--Over DesignsOver Designs
Same subject undergoes 2 or more treatmentsSame subject undergoes 2 or more treatments AdvantageAdvantage Maximizes powerMaximizes power fewest subjects neededfewest subjects needed
Limitations of reusing the same subjectLimitations of reusing the same subject May not be possibleMay not be possible
Carryover effectCarryover effect of treatmentof treatment need washoutneed washout
Length of experimentLength of experiment
Order effectOrder effect
Order should be randomized and balancedOrder should be randomized and balanced
Period effectPeriod effect
8/10/2019 Mk Select Right Stat
20/34
CrossCross--Over Designs (2)Over Designs (2)
ExamplesExamples
PrePre--post study (poor design, why?)post study (poor design, why?)
Ex) Weight before an exercise program and weightEx) Weight before an exercise program and weightafter a month of exercise programafter a month of exercise program
Traditional XTraditional X--over Studyover Study
Ex)Ex) AlternatingAlternating exposure to guided imagery procedureexposure to guided imagery procedurebetween stressful situation and a natural relaxingbetween stressful situation and a natural relaxingsituation on different days in random order andsituation on different days in random order and
assessing the effect on cravingassessing the effect on craving Stressful ImageStressful Image washout periodwashout period Relaxing ImageRelaxing Image
Relaxing ImageRelaxing Image washout periodwashout period Stressful ImageStressful Image
Ex) Drug A then cross over to BEx) Drug A then cross over to B
8/10/2019 Mk Select Right Stat
21/34
CrossCross--Over Designs (2)Over Designs (2)
Analytic MethodAnalytic Method
PrePre--post studypost study
Analyze changeAnalyze change--score or gainscore or gain--score and treat it asscore and treat it asa one sample problema one sample problem
Ex) change in weight within a person before andEx) change in weight within a person before and
after the exercise programafter the exercise program
Traditional XTraditional X--over Studyover Study
Analysis must first assess carryover effect, orderAnalysis must first assess carryover effect, ordereffect and period effect.effect and period effect.
If any effect, then must account for it.If any effect, then must account for it.
8/10/2019 Mk Select Right Stat
22/34
-level (significance level) the probabilityof claiming that there is a difference when
there is no true difference
Small
is good.
We usually set -level at 0.05. This means we allow 5% for making the
kind of error where we declare a
significant difference (reject the null
hypothesis) when the result happened by
chance (Type 1 error).
Multiple Comparison: Doing Many TestsMultiple Comparison: Doing Many Tests
8/10/2019 Mk Select Right Stat
23/34
WhenWhen 2 comparisons,2 comparisons,
(5%) should be reduced to(5%) should be reduced toadjust for the number of comparisons.adjust for the number of comparisons.
Suppose we are performing two independentSuppose we are performing two independentstatistical tests, then:statistical tests, then:
P(ofP(of rejecting the 1rejecting the 1stst when true) is 0.05when true) is 0.05
P(ofP(of rejecting the 2rejecting the 2ndnd when true) is 0.05when true) is 0.05 What is probability of rejecting at least one?What is probability of rejecting at least one?
P(ofP(of accepting 1accepting 1stst when true) is 0.95when true) is 0.95
P(ofP(of accepting 2accepting 2ndnd when true) is 0.95when true) is 0.95 Therefore,Therefore, p(ofp(of accepting both)accepting both)
= 0.95 x 0.95 = 0.9025= 0.95 x 0.95 = 0.9025
That is,That is, p(ofp(of rejecting at least one) = 0.0975rejecting at least one) = 0.0975
Multiple Comparison (2)Multiple Comparison (2)
8/10/2019 Mk Select Right Stat
24/34
Multiple Comparison (3)Multiple Comparison (3)
Number ofNumber of
independent testsindependent tests
Probability of rejectingProbability of rejecting
null hypothesis,null hypothesis,
when truewhen true
11 0.050.05
22 0.09750.097533 0.1430.143
55 0.2260.226
1010 0.4010.401
If perform enough significant tests, you are sure toIf perform enough significant tests, you are sure to
find significant results by chance alone even whenfind significant results by chance alone even whennone exists.none exists.
8/10/2019 Mk Select Right Stat
25/34
For independent tests, one easy way of adjusting theFor independent tests, one easy way of adjusting the
level of significance is to use:level of significance is to use:
0.05/k0.05/k
where k is the number of tests to be performed.where k is the number of tests to be performed.
Therefore, instead of 0.05,Therefore, instead of 0.05,
When there are 5 tests, use 0.01When there are 5 tests, use 0.01
When there are 10 tests, use 0.005When there are 10 tests, use 0.005
Multiple Comparisons: What to do? (4)Multiple Comparisons: What to do? (4)
8/10/2019 Mk Select Right Stat
26/34
Multiple Comparison (5)Multiple Comparison (5)
When testing a preWhen testing a pre--specified relationship, use aspecified relationship, use asignificance level of 5%.significance level of 5%.
When screening for interesting relationships,When screening for interesting relationships,use significance level of 1% so as not to identifyuse significance level of 1% so as not to identify
too many false relationships.too many false relationships.
8/10/2019 Mk Select Right Stat
27/34
8/10/2019 Mk Select Right Stat
28/34
MaleMale FemaleFemale
MajorMajor Number ofNumber of
ApplicantsApplicants
PercentPercent
AdmittedAdmitted
Number ofNumber of
ApplicantsApplicants
PercentPercent
AdmittedAdmittedAA 825825 62%62% 108108 82%82%
BB 560560 63%63% 2525 68%68%
CC 325325 37%37% 593593 34%34%
DD 417417 33%33% 375375 35%35%
EE 191191 28%28% 393393 24%24%
FF 373373 6%6% 341341 7%7%
TotalTotal 26912691 45%45% 18351835 30%30%
Weighted Average:Weighted Average: 39%39% 43%43%
Confounding (2)Confounding (2)
8/10/2019 Mk Select Right Stat
29/34
Example2Example2 :: Is psychiatric hospitalization rate differentIs psychiatric hospitalization rate differentin substance users versus nonin substance users versus non--users?users?
HospitalizationHospitalizationYes NoYes No
UserUser 2020 373373 5.1%
NonNon--useruser 66 316316 1.9%Substance useSubstance use looks to be associated with higherlooks to be associated with higher
psychiatric hospitalization ratepsychiatric hospitalization rate..
Separated bySeparated by Bipolar StatusBipolar Status
No BipolarNo Bipolar BBipolaripolar I/III/II
UserUser 33 176176 1.7%1.7% 1717 197197 7.9%7.9%NonNon--UserUser 44 293293 1.4%1.4% 22 2323 8.0%8.0%
Confounding (3)Confounding (3)
8/10/2019 Mk Select Right Stat
30/34
Example 3Example 3: Smoking versus MI: Smoking versus MISmokerSmoker NonNon--SmokerSmoker
MIMI 5151 5454
No MINo MI 4343 6767
54%54% 44.6%44.6% OR = 1.47OR = 1.47
MaleMale FemaleFemale
MIMI 3737 2525 1414 2929
No MINo MI 2424 2020 1919 4747
61%61% 56%56% 48%48% 38%38%OR = 1.23OR = 1.23 OR = 1.19OR = 1.19
Smokers have higher MI rate, but the magnitude of theSmokers have higher MI rate, but the magnitude of the
relative likelihood of MI (measured as odds ratio (OR)) isrelative likelihood of MI (measured as odds ratio (OR)) islarger in the combined datalarger in the combined data..
Confounding (4)Confounding (4)
8/10/2019 Mk Select Right Stat
31/34
Example 4:Example 4:1) Regression of Happiness on Smoker Group1) Regression of Happiness on Smoker Group
CoefCoef SESE pp--valuevalue
InterceptIntercept 65.0565.05 1.481.48 0.0000.000
SmokeSmoke 4.804.80 2.032.03 0.0200.020
2) Regression of Happiness on Age2) Regression of Happiness on Age
CoefCoef SESE pp--valuevalueInterceptIntercept 7.487.48 2.452.45 0.0030.003
AgeAge 1.851.85 0.070.07 0.0000.000
3) Regression of Happiness on Age and Smoke3) Regression of Happiness on Age and SmokeCoefCoef SESE pp--valuevalue
InterceptIntercept 2.652.65 2.072.07 0.2030.203
AgeAge 2.082.08 0.070.07 0.0000.000SmokeSmoke --5.255.25 0.700.70 0.0000.000
Confounding (5)Confounding (5)
8/10/2019 Mk Select Right Stat
32/34
Confounding (6)Confounding (6)
20
40
60
80
100
Happiness
Not Smoke
Without Considering Age, smokers appear to have higher mean
20
40
60
80
100
20 25 30 35 40 45Age
Y, Smoke == Not Y, Smoke == Smoke
by Smoking Status
Relationship between Happiness and Age
Increasing age is associated with greater happiness.Increasing age is associated with greater happiness.
Smokers tend to be older, making it look like smoking is associaSmokers tend to be older, making it look like smoking is associatedted
with greater happiness when not adjusting for age.with greater happiness when not adjusting for age. But smokers tend to be less happy than nonBut smokers tend to be less happy than non--smokers given same age.smokers given same age.
8/10/2019 Mk Select Right Stat
33/34
Developing
a
Statistical
Analysis
PlanDevelopingaStatisticalAnalysisPlan
Comparing two groupsComparing two groups
Continuous: tContinuous: t--testtest
Proportion: chiProportion: chi--square testsquare test Comparing multiple groups (continuous): ANOVAComparing multiple groups (continuous): ANOVA
Adjusted for other factors: ANCOVA, or regressionAdjusted for other factors: ANCOVA, or regression
Dichotomous outcome: Logistic regressionDichotomous outcome: Logistic regression Count outcome: Poisson regressionCount outcome: Poisson regression
Survival time outcome: Cox regressionSurvival time outcome: Cox regression
Watch for correlated data (repeated measures, clustersWatch for correlated data (repeated measures, clusters e.g., teeth in the mouthe.g., teeth in the mouth
8/10/2019 Mk Select Right Stat
34/34
To Keep in MindTo Keep in Mind
Typically, multiple appropriate methods are availableto analyze the same data that could yield legitimateanswers.
Try to use at least two different available methods toconfirm your results.
Always look at the raw data and display datagraphically, so learn to choose the right graphical
displays (ex: cross tabs, scatter plots, box plots)
It helps to make sample tables summarizing results
before you start the analysis.