Hypothesis testing & Inferential Statistics

Hypothesis Testing

• The process of determining whether a hypothesis is supported by the results of the study.

• Use inferential statistics to draw conclusions (make inferences) about the population based on data collected from a sample.

• Ex: cholesterol levels while on “an all fruit diet” vs. no diet at all. – Hypothesis: individuals in the all fruit diet will have lower levels of

cholesterol than general population.

Null and Alternate Hypothesis

• Goal of science is to reject untrue information, thereby the information that is left over is assumed to be true. – Statistically impossible to show that something is true. – Statistically possible to show that something is false.

• Counter intuitive rationale– we must reject a hypothesis as false in order to find support for the

hypothesis we are seeking.

Null Hypothesis

• Ex: we want to show that an all fruit diet lowers cholesterol compared to no diet at all.

• Null hypothesis: no difference between the groups being compared. – H0 : μ 0 = μ 1

– H0 : μ of all fruit = μ of general population

• Goal of research study is to determine whether a null hypothesis is true or false. – If the null hypothesis is rejected, then a possible true result remains.

Alternate Hypothesis

• Ex: we want to show that an all fruit diet lowers cholesterol compared to no diet at all.

• Alternate (or research) hypothesis: there is a significant difference between the groups being tested. – Ha : μ 0 < μ 1

– Ha : μ of all fruit < μ of general population

• Goal of research study is to support the alternate hypothesis by rejecting the null hypothesis.

Directional hypothesis or One-tailed

• Directional hypothesis (one-tailed hypothesis)– The experimenter predicts the direction of the expected

results. – Alternate directional hypothesis

– Ha : μ 0 < μ 1

– Ha : μ of all fruit < μ of general population

– Null directional hypothesis= H0 : μ 0 ≥μ 1

– H0 : μ of all fruit ≥ μ of general population

Nondirectional hypothesis or Two-tailed

• Nondirectional hypothesis (two-tailed hypothesis)– The experimenter predicts differences between the

groups, but is unsure what the differences will be.

– Alternate directional hypothesis: Ha : μ 0 ≠ μ 1

• Ha : μ of all fruit ≠ μ of general population

– Null directional hypothesis= H0 : μ 0 = μ 1

• H0 : μ of all fruit = μ of general population

Type I Error

• Sometimes we can make mistakes in our research.

• At times, we reject the null hypothesis when it should NOT have been rejected.

• Ex: we say the all fruit diet lowers cholesterol when it actually does not.

• Also known as a false positive: we say there was a difference between groups, when in reality there was no difference.

• We found a difference, but it was due to chance and the results are a fluke.

Type II Error

• In some cases, the null hypothesis SHOULD be rejected, but we fail to find a difference between the groups.

• The null hypothesis is false, but accept it anyway.

• Ex: we say the 2 groups have equal levels of cholesterol when in fact the all-fruit group has lower levels.

• Somehow, we failed to find a difference between the 2 groups.

Type I vs. Type II errors

• Type I error:– Null hypothesis: not pregnant– Alternative hypothesis: pregnant– If we reject the null (not pregnant) but we are incorrect, she is going

to think she is pregnant when she is really not.

– Leads to actions:• Freaking out• Happiness – tell friends• Buy baby clothes

Type I vs. Type II errors

• Type II error:– Null hypothesis: not pregnant– Alternative hypothesis: pregnant– if we fail to reject the null, we keep the null and she will think she is

not pregnant when she really is,

– Leads to no actions:• no prenatal care• May go drinking and hurt the fetus.

Type I vs. Type II

• Which is worse for research?

• Type I: saying a result is true when it is not true is more detrimental to research.

• But in other cases, maybe Type II can be worse.

• In order to avoid both errors, researchers try to replicate the results.

Statistical Significance

• Ex: we find that the all-fruit diet group has lower levels of cholesterol than the rest of the population. – Significantly lower!

• Statistical significance at .05 level (p ≤.05)– We will get these results by chance only 5 times or less out of 100. – 95% of the time, these results are due to our manipulation– We can reject the null hypothesis because the pattern of the data are

unlikely to have occurred by chance. – probability of making a Type I error: 5 times or less out of 100– The field has established the alpha level at .05

Single-group design

• Research involving only one group and no control group. – Simplest kind of hypothesis testing

• We compare the results of the group (sample) with the performance of the general population.

• Use parametric tests, a type of inferential statistics, that requires certain parameters about the population (i.e., mean, standard deviation)– t test

Single sample t-test

• Parametric inferential statistical test of the null hypothesis.

• Used for a single sample when we know the population mean, but not the population standard deviation.

• T-tests compare differences between the mean of a sample and the mean of a population. – However, we need to compare the sample mean with a distribution of

sample means

Sampling Distribution

• Distribution of sample means based on random samples, of fixed sizes, from a population.

• Ex: IQ scores of 1000 people (i.e., population)– Take 100 samples of 10 people, plot the mean IQ of each sample. – The mean of the population IQ will equal the mean of the distribution

of means.

Sampling Distribution

• Distribution of scores (pop.) and distribution of means have the same mean.

• The standard deviation (SD) of distribution of scores is bigger than the SD distribution of means.

Sampling Distribution

• The standard deviation of a distribution of sample means has a new name.

• Standard error of the mean: the standard deviation of a distribution of means.

• sM = s

√N

• Where √ s = ∑( X - X ) ² _________ N - 1

T-test• To calculate a t-test, we need to know the sample mean,

the population mean, and the standard error of the mean.

t = (X - µM) sM

• s = 2.97• Pop. mean = 11• t = 1.05

• Our sample mean falls +1.05 standard deviations above the pop. mean.

• Is this difference large enough to be statistically significant?

T-distributions

• We compare the t-value to a standardized distribution of t-values (i.e., t distributions).

• As the sample size increases, the t-distributions approaches a normal distribution.

• In t-distributions, we base sample size in terms of degrees of freedom or df

Degrees of freedom

• Degrees of freedom– Number of scores in a sample that are free to vary.

• Ex: 2, 5, 6, 9, 11, 15 mean = 8– In order to maintain the mean of 8, five scores are free to vary, except

for the last one.

• df = N - 1

T-distributions

One-tailed t-Test

• We compare our t-value to a distribution of t-values.

• If our t-value is larger than the cut-off, we reject the null hypothesis.

• Directional hypothesis

– Ha : μ of all fruit < μ of general population

– H0 : μ of all fruit ≥ μ of general population

Two-tailed t-test

• If t-value falls in the region of rejection, we reject the null hypothesis.

• Nondirectional hypothesis– Ha : μ of all fruit ≠ μ of general population

– H0 : μ of all fruit = μ of general population

T-table (appendix A.3)

• Which test is more conservative, one-tailed or two-tailed? – Two-tailed test because it is more difficult to beat the critical value to

reject the null hypothesis.

Statistical Power

• Probability that we can reject the null hypothesis and find significant differences when differences truly exist.

• To increase power– Use one-tailed tests– Increase the sample size: as sample size increases, the critical value

decreases (i.e., easier to reject the null hypothesis when the null is false)

Steps to hypothesis testing

• The t test– The six steps of hypothesis testing

• 1. Identify mean of sample and mean of population of sample means.

• 2. State the hypotheses (null and alternate)• 3. Characteristics of the comparison distribution

– Find standard error of the mean• 4. Critical values• 5. Calculate t-value• 6. Decide to reject or sustain the null hypothesis.