Review of observational study Review of observational study design and basic statistics for design and basic statistics for

contingency tablescontingency tables

According to scientists, too much coffee may cause... 1986 --phobias, --panic attacks 1990 --heart attacks, --stress, --osteoporosis 1991 -underweight babies, --hypertension 1992 --higher cholesterol 1993, 08 --miscarriages 1994 --intensified stress 1995 --delayed conception But scientists say coffee also may help prevent... 1988 --asthma 1990 --colon and rectal cancer,... 2004—Type II Diabetes (*6 cups per day!) 2006—alcohol-induced liver damage 2007—skin cancer

Coffee Chronicles BY MELISSA AUGUST, ANN MARIE BONARDI, VAL CASTRONOVO, MATTHEW

JOE'S BLOWS Last week researchers reported that coffee might help prevent Parkinson's disease. So is the caffeine bean good for you or not? Over the years, studies haven't exactly been clear:

Medical StudiesMedical Studies

Evaluate whether a risk factor (or preventative factor) increases (or decreases) your risk for an outcome (usually disease, death or intermediary to disease).

The General Idea…

Exposure Disease?

Observational vs. Observational vs. Experimental StudiesExperimental Studies

Observational studies – the population is observed without any interference by the investigator

Experimental studies – the investigator tries to control the environment in which the hypothesis is tested (the randomized, double-blind clinical trial is the gold standard)

Limitation of observational Limitation of observational research: confoundingresearch: confounding

Confounding: risk factors don’t happen in isolation, except in a controlled experiment. – Example: In a case-control study of a salmonella outbreak,

tomatoes were identified as the source of the infection. But the association was spurious. Tomatoes are often eaten with serrano and jalapeno peppers, which turned out to be the true source of infection.

– Example: Breastfeeding has been linked to higher IQ in infants, but the association could be due to confounding by socioeconomic status. Women who breastfeed tend to be better educated and have better prenatal care, which may explain the higher IQ in their infants.

Confounding: A major problem Confounding: A major problem for observational studiesfor observational studies

Exposure Disease

Confounder

?

Why Observational Studies?Why Observational Studies?

CheaperFasterCan examine long-term effectsHypothesis-generatingSometimes, experimental studies are not

ethical (e.g., randomizing subjects to smoke)

Possible Observational Possible Observational Study DesignsStudy DesignsCross-sectional studies

Cohort studies

Case-control studies

Cross-Sectional (Prevalence) Cross-Sectional (Prevalence) StudiesStudies

Measure disease and exposure on a random sample of the population of interest. Are they associated?

Marginal probabilities of exposure AND disease are valid, but only measures association at a single time point.

The 2x2 TableThe 2x2 Table

  Exposure (E) No Exposure (~E)

 

Disease (D) a b (a+b)/T = P(D)

No Disease (~D) c d (c+d)/T = P(~D)

  (a+c)/T = P(E) (b+d)/T = P(~E)

Marginal probability of disease

Marginal probability of exposure

N

Example: cross-sectional Example: cross-sectional studystudy

Relationship between atherosclerosis and late-life depression (Tiemeier et al. Arch Gen Psychiatry, 2004).

Methods: Researchers measured the prevalence of coronary artery calcification (atherosclerosis) and the prevalence of depressive symptoms in a large cohort of elderly men and women in Rotterdam (n=1920).

Example: cross-sectional Example: cross-sectional studystudy

P(“D”)= Prevalence of depression (sub-thresshold or depressive disorder): (20+13+12+9+11+16)/1920 = 4.2%

P(“E”)= Prevalence of atherosclerosis (coronary calcification >500):(511+12+16)/1920 = 28.1%

The 2x2 table:The 2x2 table:   

Coronary calc >500539

Coronary calc <=500 1381

  81 1839 1920

Any depression

None

28 511

53 1328

P(depression)= 81/1920 = 4.2%

P(atherosclerosis) = 539/1920 = 28.1%

P(depression/atherosclerosis) = 28/539 = 5.2%

Difference of proportions Z-test:Difference of proportions Z-test:

   

Coronary calc >500539

Coronary calc <=500 1381

  81 1839 1920

Any depression

None

28 511

53 1328

038.1381

53;052.

539

28// unblockeddepressionrosisatheroscledepression pp

18.;33.10101.

014.

1381)042.1)(042(.

539)042.1)(042(.

038.052.

).(.

pdifferencees

differenceZ

   

Coronary calc >500539

Coronary calc <=500 1381

  81 1839 1920

Any depression

None

28 511

53 1328

2.19) (0.86, CI 95% ;37.1038.

052.RR

Or, use relative risk (risk ratio):Or, use relative risk (risk ratio):

Interpretation: those with coronary calcification are 37% more likely to have depression (not significant).

Or, use chi-square test:Or, use chi-square test:   

Coronary calc >500539

Coronary calc <=500 1381

  81 1839 1920

Any depression None

28 511

53 1328

Observed:

Expected:   

Coronary calc >500539

Coronary calc <=500 1381

  81 1839 1920

Any depression None

539*81/1920=

22.7

539-22.7=

516.381-22.7=

58.31381-58.3=

1322.7

Chi-square test:Chi-square test:

expected

expected) - (observed 22

18.

77.17.1322

)7.13221328(

3.516

)3.516511(

3.58

)3.5853(

7.22

)7.2228(

2

2222

1

p

Note: 1.77 = 1.332

Chi-square test also works for Chi-square test also works for bigger contingency tables (RxC):bigger contingency tables (RxC):

Chi-square test also works for Chi-square test also works for bigger contingency tables (RxC):bigger contingency tables (RxC):

Coronary calcification No

depression

Sub-threshhold depressive symptoms

Clinical

depressive disorder

0-100 865 20 9

101-500 463 13 11

>500 511 12 16

Coronary calcification

No depression

Sub-threshhold depressive symptoms

Clinical

depressive disorder

0-100 865 20 9 894

101-500 463 13 11 487

>500 511 12 16 539

1839 45 36 1920

Observed: Expected:

Coronary calcification No

depression

Sub-threshhold depressive symptoms

Clinical

depressive disorder

0-100 894*1839/1920=

856.3

849*45/1920=

21

894-(21+856.3)=16.7

101-500 487*1839/1920=

466.5

487*45/1920=

11.4

487-(466.5+11.4)=9.1

>500 1839-(856.3+466.5)=

516.2

45-(21+11.4)=

12.6

36-(16.7+9.1)=

10.2

Chi-square test:Chi-square test:

expected

expected) - (observed 22

096.

877.72.10

)2.1016(

6.12

)6.1212(

2.516

)2.516511(

1.9

)1.911(

4.11

)4.1113(

5.466

)5.466463(

7.16

)7.169(

21

)2120(

3.856

)3.856865(

222

222

2222

4

p

Cause and effect?Cause and effect?

atherosclerosis

depression in elderly

?Biological changes

?Lack of exercise Poor Eating

Confounding?Confounding?

atherosclerosis

depression in elderly

Advancing Age

?Biological changes

?Lack of exercise Poor Eating

Cross-Sectional StudiesCross-Sectional Studies

Advantages: – cheap and easy– generalizable– good for characteristics that (generally) don’t change

like genes or gender

Disadvantages – difficult to determine cause and effect– problematic for rare diseases and exposures

2. Cohort studies2. Cohort studies::

Sample on exposure status and track disease development (for rare exposures)

Marginal probabilities (and rates) of developing disease for exposure groups are valid.

Example: The Framingham Example: The Framingham Heart StudyHeart Study

The Framingham Heart Study was established in 1948, when 5209 residents of Framingham, Mass, aged 28 to 62 years, were enrolled in a prospective epidemiologic cohort study.

Health and lifestyle factors were measured (blood pressure, weight, exercise, etc.).

Interim cardiovascular events were ascertained from medical histories, physical examinations, ECGs, and review of interim medical record.

Example 2: Johns Hopkins Precursors StudyExample 2: Johns Hopkins Precursors Study(medical students 1948 through 1964)(medical students 1948 through 1964)

http://www.jhu.edu/~jhumag/0601web/study.html

From the John Hopkin’s Magazine website (URL above).

Cohort StudiesCohort Studies

Target population

Exposed

Not Exposed

Disease-free cohort

Disease

Disease-free

Disease

Disease-free

TIME

  Exposure (E) No Exposure (~E)

 

Disease (D) a b

No Disease (~D) c d

  a+c b+d

)/()/(

)~/(

)/(

dbbcaa

EDP

EDPRR

risk to the exposed

risk to the unexposed

The Risk Ratio, or Relative Risk (RR)

400 400

1100 2600

0.23000/4001500/400 RR

Hypothetical DataHypothetical Data

  Normal BP

Congestive Heart Failure

No CHF

1500 3000

High Systolic BP

Advantages/Limitations:Advantages/Limitations:Cohort StudiesCohort Studies

Advantages:– Allows you to measure true rates and risks of disease for the

exposed and the unexposed groups.– Temporality is correct (easier to infer cause and effect).– Can be used to study multiple outcomes. – Prevents bias in the ascertainment of exposure that may occur

after a person develops a disease. Disadvantages:

– Can be lengthy and costly! 60 years for Framingham.– Loss to follow-up is a problem (especially if non-random).– Selection Bias: Participation may be associated with exposure

status for some exposures

Case-Control StudiesCase-Control Studies

Sample on disease status and ask retrospectively about exposures (for rare diseases) Marginal probabilities of exposure for cases and

controls are valid.

• Doesn’t require knowledge of the absolute risks of disease

• For rare diseases, can approximate relative risk

Target population

Exposed in past

Not exposed

Exposed

Not Exposed

Case-Control StudiesCase-Control Studies

Disease

(Cases)

No Disease

(Controls)

Example: the AIDS epidemic Example: the AIDS epidemic in the early 1980’sin the early 1980’s

Early, case-control studies among AIDS cases and matched controls indicated that AIDS was transmitted by sexual contact or blood products.

In 1982, an early case-control study matched AIDS cases to controls and found a positive association between amyl nitrites (“poppers”) and AIDS; odds ratio of 8.6 (Marmor et al. 1982). This is an example of confounding.

Case-Control Studies in Case-Control Studies in HistoryHistory

In 1843, Guy compared occupations of men with pulmonary consumption to those of men with other diseases (Lilienfeld and Lilienfeld 1979).

Case-control studies identified associations between lip cancer and pipe smoking (Broders 1920), breast cancer and reproductive history (Lane-Claypon 1926) and between oral cancer and pipe smoking (Lombard and Doering 1928). All rare diseases.

Case-control studies identified an association between smoking and lung cancer in the 1950’s.

Case-control exampleCase-control example

A study of the relation between body mass index and the incidence of age-related macular degeneration (Moeini et al. Br. J. Ophthalmol, 2005).

Methods: Researchers compared 50 Iranian patients with confirmed age-related macular degeneration and 80 control subjects with respect to BMI, smoking habits, hypertension, and diabetes. The researchers were specifically interested in the relationship of BMI to age-related macular degeneration.

ResultsResults

Table 2  Comparison of body mass index (BMI) in case and control groups

Case n = 50(%) Control n = 80 (%) p Value

Lean BMI <20 7 (14) 6 (7.5) NS

Normal 20   BMI <25 16 (32) 20 (25) NS

Overweight 25   BMI <30 21 (42) 36 (45) NS

Obese BMI  30 6 (12) 18 (22.5) NS

NS, not significant.

  Overweight Normal  

ARMD 27 23

No ARMD 54 26

 

Corresponding 2x2 Table

What is the risk ratio here?

Tricky: There is no risk ratio, because we cannot calculate the risk of disease!!

50

80

The odds ratio…The odds ratio…

We cannot calculate a risk ratio from a case-control study.

BUT, we can calculate a measure called the odds ratio…

Odds vs. RiskOdds vs. Risk

If the risk is… Then the odds are…

½ (50%)

¾ (75%)

1/10 (10%)

1/100 (1%)

Note: An odds is always higher than its corresponding probability, unless the probability is 100%.

1:1

3:1

1:9

1:99

The proportion of cases and controls are set by the investigator; therefore, they do not represent the risk (probability) of developing disease.

bc

ad

dcba

dcddccbabbaa

ORDEP

DEP

DEPDEP

)/()/()/()/(

)~/(~)~/(

)/(~)/(

  Exposure (E) No Exposure (~E)

 

Disease (D) a b

No Disease (~D) c d

 

The Odds Ratio (OR)

a+b=cases

c+d=controls

Odds of exposure in the cases

Odds of exposure in the controls

dbca

dcba

bc

adOR

  Exposure (E) No Exposure (~E)

 

Disease (D) a b

No Disease (~D) c d

 

The Odds Ratio (OR)

Odds of disease for the exposed

Odds of exposure for the controls

Odds of exposure for the cases.

Odds of disease for the unexposed

=

Odds of exposure in the controls

Odds of exposure in the cases

Bayes’ Rule

Odds of disease in the unexposed

Odds of disease in the exposed

What we want!

)~/(~

)~/()/(~

)/(

DEP

DEPDEP

DEP

)(~

)(~)~/(~)(~

)()/(~)(

)(~)~/()(

)()/(

DP

EPEDPDP

EPEDPDP

EPEDPDP

EPEDP

)~/(~

)~/()/(~

)/(

EDP

EDPEDP

EDP

Proof via Bayes’ Rule (optional)

dbca

dcba

bc

adOR

  Overweight Normal  

ARMD a b

No ARMD c d

 

The Odds Ratio (OR)

Odds of ARMD for the overweight

Odds of overweight for the controls

Odds of overweight for the cases.

Odds of ARMD for the normal weight

57.54*23

26*27

26542327

OR

  Overweight Normal  

ARMD 27 23

No ARMD 54 26

 

The Odds Ratio (OR)

57.54*23

26*27

26542327

OR

  Overweight Normal  

ARMD 27 23

No ARMD 54 26

 

The Odds Ratio (OR)

Can be interpreted as: Overweight people have a 43% decrease in their ODDS of age-related macular degeneration. (not statistically significant here)

The odds ratio is a good The odds ratio is a good approximation of the risk ratio approximation of the risk ratio

if the disease is rare.if the disease is rare.

RROR If the disease is rare (affecting <10% of the population), then:

WHY?

If the disease is rare, the probability of it NOT happening is close to 1, and the odds is close to the risk. Eg:

50.10:1

20/1

474.9/1

19/1

RR

OR

The rare disease assumptionThe rare disease assumption

RROR EDPEDP

EDPEDP

EDPEDP

)~/()/(

)~/(~)~/(

)/(~)/(

1

1

When a disease is rare: P(~D) = 1 - P(D) 1

The odds ratio vs. the risk ratioThe odds ratio vs. the risk ratio

1.0 (null)

Odds ratio

Risk ratio Risk ratio

Odds ratio

Odds ratio

Risk ratio Risk ratio

Odds ratio

Rare Outcome

Common Outcome

1.0 (null)

When is the OR is a good When is the OR is a good approximation of the RR?approximation of the RR?

General Rule of Thumb:

“OR is a good approximation as long

as the probability of the outcome in the

unexposed is less than 10%”

Prevalence of age-related macular degeneration is about 6.5% in people over 40 in the US (according to a 2011 estimate). So, the OR is a reasonable approximation of the RR.

Advantages/Limitations:Advantages/Limitations:Case-control studiesCase-control studies

Advantages:– Cheap and fast– Efficient for rare diseases

Disadvantages:– Getting comparable controls is often tricky– Temporality is a problem (did risk factor cause disease

or disease cause risk factor?– Recall bias

Inferences about the odds Inferences about the odds ratio…ratio…

Properties of the OR (simulation)(50 cases/50 controls/20% exposed)

If the Odds Ratio=1.0 then with 50 cases and 50 controls, of whom 20% are exposed, this is the expected variability of the sample ORnote the right skew

Properties of the lnOR

dcba

1111

Standard deviation =

Hypothetical DataHypothetical Data

0.8)10)(6(

)24)(20(OR

25.8) - (2.47(8.0)e ,(8.0)e CI %95 24

1

10

1

6

1

20

196.1

24

1

10

1

6

1

20

196.1

  Amyl Nitrite Use No Amyl Nitrite

 

AIDS 20 10

Does not have AIDS

6 24

 

30

30

Note that the size of the smallest 2x2 cell determines the magnitude of the variance

When can the OR mislead?When can the OR mislead?

Example:Example:Does dementia predict death?Does dementia predict death?Dementia: The leading predictor of death in

a defined elderly population. Neurology 2004; 62: 1156-1162

Among patients with dementia: 291/355 (82%) died

Among patients without dementia: 947/4328 (22%) died

Dementia studyDementia study

Authors report OR = 16.23 (12.27, 21.48)But the RR = 3.72 Fortunately, they do not dwell on the OR,

but it could mislead if not interpreted correctly…

Better to give OR or RR?Better to give OR or RR?From an RCT (prospective!) of a new diet drug, the authors

showed the following table:

Odds Ratios for losing at least 5kg were:4.0 (low dose vs. placebo)

20.9 (medium dose vs. placebo)31.5 (high dose vs. placebo)

Better to give OR or RR?Better to give OR or RR?

Corresponding RRs are:59%/29%=2 (low dose vs. placebo)

87%/29%=3 (medium dose vs. placebo)91%/29%=3 (high dose vs. placebo)

Summary of statistical tests Summary of statistical tests for contingency tablesfor contingency tables

Table Size Test or measures of association

2x2 risk ratio (cohort or cross-sectional studies)

odds ratio (case-control studies)

Chi-square

difference in proportions

Fisher’s Exact test (cell size less than 5)

RxC Chi-square

Fisher’s Exact test (expected cell size <5)

Fisher’s Exact TestFisher’s Exact Test

Fisher’s “Tea-tasting Fisher’s “Tea-tasting experiment”experiment”

Claim: Fisher’s colleague (call her “Cathy”) claimed that, when drinking tea, she could distinguish whether milk or tea was added to the cup first.

To test her claim, Fisher designed an experiment in which she tasted 8 cups of tea (4 cups had milk poured first, 4 had tea poured first).

Null hypothesis: Cathy’s guessing abilities are no better than chance.

Alternatives hypotheses:

Right-tail: She guesses right more than expected by chance.

Left-tail: She guesses wrong more than expected by chance

Fisher’s “Tea-tasting Fisher’s “Tea-tasting experiment”experiment”

Experimental Results:

  Milk Tea  

Milk 3 1

Tea 1 3

 

Guess poured first

Poured First

4

4

Fisher’s Exact TestFisher’s Exact TestStep 1: Identify tables that are as extreme or more extreme than what actually happened:

Here she identified 3 out of 4 of the milk-poured-first teas correctly. Is that good luck or real talent?

The only way she could have done better is if she identified 4 of 4 correct.

  Milk Tea  

Milk 3 1

Tea 1 3

 

Guess poured firstPoured First

4

4

  Milk Tea  

Milk 4 0

Tea 0 4

 

Guess poured firstPoured First

4

4

Fisher’s Exact TestFisher’s Exact TestStep 2: Calculate the probability of the tables (assuming fixed marginals)

  Milk Tea  

Milk 3 1

Tea 1 3

 

Guess poured first

Poured First

4

4

  Milk Tea  

Milk 4 0

Tea 0 4

 

Guess poured first

Poured First

4

4

229.)3(

84

41

43 P

014.)4(

84

40

44 P

Step 3: to get the left tail and right-tail p-values, consider the probability mass function:

Probability mass function of X, where X= the number of correct identifications of the cups with milk-poured-first:

229.)3(

84

41

43 P

014.)4(

84

40

44 P

514.)2(

84

42

42 P

229.)1(

84

43

41 P

014.)0(

84

44

40 P

“right-hand tail

probability”: p=.243

“left-hand tail probability” (testing the alternative

hypothesis that she’s

systematically wrong): p=.986

SAS also gives a “two-sided p-value” which is calculated

by adding up all probabilities in the distribution that are less than or equal to

the probability of the observed table (“equal or more extreme”). Here:

0.229+.014+.0.229+.014= .4857

Summary of statistical tests Summary of statistical tests for contingency tablesfor contingency tables

Table Size Test or measures of association

2x2 risk ratio (cohort or cross-sectional study)

odds ratio (case-control study)

Chi-square

difference in proportions

Fisher’s Exact test (cell size less than 5)

RxC Chi-square

Fisher’s Exact test (expected cell size <5)