Comparing Two Population Means. Two kinds of studies or experiments There are two general research strategies that can be used to compare the two populations

Comparing Two Population Means

Two kinds of studies or experimentsThere are two general research strategies that can be

used to compare the two populations of interest:• The two samples could be drawn from two

independent populations (e.g. women and men, or patients taking drug A vs. those taking drug B)between subjects or independent samples design

• The two sets of data could come from related populations (e.g. “before treatment” and “after treatment”, patient taking drug A and the SAME patient taking drug B at a later date, or subjects in different treatment groups are meaningfully matched according to some criteria)within subjects or dependent samples/matched pairs design

We will focus on the independent samples case first.

Comparing Two Population Means: Using Independent Samples

OBSERVATIONAL STUDY or EXPERIMENT?

Observational Comparative Study

• A research study in which two or more groups are compared with respect to some measurement or response.

• The groups, determined by their natural characteristics, are merely “observed.”

Observational Comparative Study

Population 1&

Population 2&

vs.

11 and sX

Sample size = n1

Calculate:

Sample size = n2

Calculate: 22 and sX

To make inferences use:

Hypothesis test, CI for difference in means, effect size (d)

Example 1: Normal Human Body Temperature (females vs. males)

• Is the normal human body temperature the same for females and males?

• To answer this question two samples of size 65 are independently drawn from these two populations.

• Let’s assume that our research hypothesis is that, for what ever reason, females have a higher mean body temperature than males.

Comparing Two Population Means: Using Independent Samples

Comparative Experiment

• A study in which two or more groups (see later) are randomly assigned to a “treatment” to see how the treatment affects some response.

• If each experimental unit has the same chance of receiving any treatment, then the experiment is called a completely randomized design.

Comparative Experiment

Treatment 1&

Treatment 2&

vs.

11 and sX

Sample size = n1

Calculate:

Sample size = n2

Calculate: 22 and sX

Population Randomly assign

To make inferences use:

Hypothesis test, CI for difference in means, effect size (d)

All possible questions in statistical notation

In general, we can always compare two means by seeing how their difference ( – compares to 0:

This comparison… is equivalent to …

1 2 1 - 2 0

1 > 2 1 - 2 > 0

1 < 2 1 - 2 < 0

Hypotheses

For testing equality

Ho: 1 = 2 or (1 – 2) = 0

The possible alternatives are:

HA: 1 > 2 or (1 – 2) > 0 (upper-tail)

HA: 1 < 2 or (1 – 2) < 0 (lower-tail)

HA: 1 2 or (1 – 2) 0 (two-tailed)

Note: If we wanted to establish that one mean was say e.g. at least 10 units larger than the other we could replace 0 in these statements by 10. In general to establish a difference of at least units then we replace 0 by .


Recall our research hypothesis is the females have a higher mean body temperature than males, therefore we have…

F = mean body temperature for females

M= mean body temperature for males

Ho: F = M or equivalently (F – M) = 0

HA: F > M or equivalently (F – M) > 0

Test StatisticThe basic form of the two-sample t test statistic is...

means samplein differencetheoferrorstandarddifferenceedhypothesizmeanssampleindifferencet

which assuming the following assumptions are satisfied has an approximate t-distribution (df

(see 3 below)).

1) Both populations are approximately normally distributed. This assumption can be relaxed when both sample sizes (n1 and n2) are “large”.

2) Random, independent samples were drawn from the two populations of interest.

3) The df depends on how the standard error of the difference is estimated.

Test Statistic

)( ~

21

21

means samplein differencetheoferrorstandarddifferenceedhypothesizmeanssampleindifference

dfondistributitXXSE

XXt

t

The standard error of the difference in sample means is calculated to different ways depending on whether or not we assume the population variances are equal.

i.e., Can we assume ?22

21

Yes, use pooled t-test

No, use Welch’s t-test

Assuming equal population variances (pooled t-test)

Estimate the standard error of the difference using the common pooled variance :

21

n1

n1s)

21( 2

pXXSE

Then the sampling distribution is a t-distribution with n1+n2-2 degrees of freedom (df).

where

Rule O’ Thumb:Assume variances are equal only if neither sample standard deviation is more than twice that of the other sample standard deviation.

2nn1)s(n1)s(n

s21

2211p

222

Estimate of the common variance (2)

If variances of the measurements of the two groups are not equal (Welch’s t-test)...

Estimate the standard error of the difference as:

2

2

1

1ns

ns

21

22

XXSE

Then the sampling distribution is an approximate t distribution with a complicated formula for d.f.

11 2

2

2

22

1

2

1

21

2

2

22

1

21

n

ns

n

ns

ns

ns

df Always round down!

Quantifying the Size of the Difference in Population Means (1 – 2)

To quantify the size of the effect, i.e. the difference in the population means we use…

• Confidence Interval for (1 – 2)

(estimate) + (table value) SE(estimate) basic form

• Effect Size (d)

)()()( 2121 XXSEtabletXX

ps

XXd 21


STEP 1) State Hypotheses

F = mean body temperature for females

M= mean body temperature for males

Ho: F = M or equivalently (F – M) = 0

HA: F > M or equivalently (F – M) > 0


STEP 2) Determine Test Criteria

a) Choose as a Type I Error is of little consequence.

b) Use two-sample t-test, either pooled t-test or Welch’s t-test. As to which form to use we need to examine our data in terms of the equality of population variances. If uncertain use Welch’s!


STEP 3) Collect Data and Compute Test Statistic

Take independent samples from the two populations and examine the resulting data.


Populations appear normally distributed and our sample sizes are “large” (> 30).

Few mild outliers for in sample for females

Sample mean for females appears to slightly larger than that for males

Variation appears to be similar for both samples.

The sample standard deviation for females (sF = .74) is larger than that for males (sM = .70) although it is not twice as large, thus we assume pop. variances are equal.

Oneway Analysis > Means/Anova/Pooled t


STEP 3) Collect Data and Compute Test Statistic

Because it seems reasonable to assume the population variances are approximately equal we will use a pooled t-test.

Pooled t-test calculations (FYI)

For the body temperature example

Assuming equal variances, the pooled estimate of common variance is

So the standard error of the difference in sample means is

oM

oF

oM

oF

ss

XX

70. 74.

10.98 39.98

52.26565

)70)(.165()74)(.165(

2

)1()1( 22222

MF

MMFFp nn

snsns

127.65

1

65

152.

11)(

21

221

nnsXXSE p

Pooled t-test calculations (FYI)

For the body temperature example

Computing test statistic gives

t = 2.28

P-value = .0121


STEP 4) Compute p-value The p-value = .0121, which indicates we have a

1.21% chance of observing a difference in sample means this large by chance variation alone if in fact the population means were equal.

STEP 5) Make Decision and Interpret

Because p-value < .05 we reject Ho and conclude that the mean normal body temperature for females is larger than that for males.


STEP 6) Quantify Significant Findings

Construct a 95% CI for (F – M)

(98.39 – 98.10) + (1.98)(.127) = (.039o , .541o)

We estimate the normal mean body temperature for women is between .039o F to .541o F larger than the normal mean body temperature for men.

)()()( 2121 XXSEtabletXX


STEP 6) Quantify Significant FindingsEffect Size (d)

Thus effect size is moderate at best with % overlap of the two body temperature distributions being around 72.6%. Furthermore, the absolute difference represented by the lower confidence limit (LCL = .039 deg F) hardly seems of any physiological importance.

402.721.

10.9839.9821

ps

XXd

Power and Sample Size Issues(equal variance case)

Power is a function of:

• The sample sizes (n1 and n2), as the sample sizes increase so does power.

• The smaller the common variance the larger the power.

• The larger the difference () to detect the greater the power.

• As always, as increases the power increases.

Power and Sample Size Issues

Common standard deviation to both populations/groups

= difference in population means which makes alternative true

Common sample size (n1 = n2 = n)

Probability of rejecting Ho when difference in means is


Error Std. Dev = common standard deviation () = .721 for body temperature example.

Difference in Means = = |1 – 2| = .29 which is the difference in the samples for body temperature example.

Alpha = P(Type I Error) = .05

For body temperature example we have a power of 1 – .6240.


Error Std. Dev = common standard deviation () = .721 for body temperature example.

Difference in Means = = |1 – 2| = .29 which is the difference in the samples for body temperature example.

Alpha = P(Type I Error) = .05

Example 3: Gestational age of babies born to women with preeclampsia

We wish to compare the mean gestational age (in weeks) of babies born to women with preeclampsia during pregnancy vs. those who had normal pregnancies.

Data:Preeclampsia: 38, 32, 42, 30, 38,35, 32, 38, 39, 29, 29, 32Normal: 40, 41, 38, 40, 40, 39, 39, 41, 41, 40, 40, 40


The SD for the preeclamptic group is over 4 times larger!!

The sample standard deviations control the slope of the reference lines, the line for preeclamptics is over 4 times steeper.


Formally testing equality of variances:

Test Statistic:

22

22

:

:

normaliapreeclampsA

normaliapreeclampso

H

H

ly.respective

1 dfr denominato

1 dfnumerator

1 dfr denominato

1dfnumerator

with~ , max

2

1

1

2

22

21

21

22

n

n

n

n

ondistributiFs

s

s

sF


F = (4.40)2/(.900)2 = 23.89

i.e. the sample variance for the preeclamptic group is 23.89 times larger than the sample variance for the normal pregnancy group. The p-value comes from an a F-dist. with numerator df = 11, denominator df = 11.

We have strong evidence against equality of the variance of the response for these two populations of women (p < .0001) therefore we should not use a pooled t-Test!

F-distribution

Then the P-value < .0001

When H0 is true, the F-ratio ~ F (df1,df2)

Let’s say our observed value for F was F0 = 23.89

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

F-distribution

For example, consider the F-distribution with 11 and 11 df


Oneway Analysis > t Test (i.e. non-pooled t–Test)

We strong evidence that the mean gestational age of babies born to preeclamptic mothers is lower than that for babies born to women with normal pregnancies (p = .0013).

Furthermore, we estimate that the mean gestational age for babies born to preeclamptic mothers is between 2.59 and 8.24 weeks less than the mean gestational age for babies born to mothers with normal pregnancies.

Summary• If in doubt, use non-pooled t-Test.

• Be sure to quantify significant differences with a CI for (1 – 2) or other measure of effect size.

• Assess normality of both samples. When sample sizes are small, non-normality presents a problem and we should use test procedures that do not require normality (i.e. nonparametric tests, we’ll see these later).

Documents

Comparing Two Population Means. Two kinds of studies or experiments There are two general research strategies that can be used to compare the two populations