Inferential Statistics

Testing for DifferencesTesting for Differences

Introduction

Whether the research design is Whether the research design is experimental, quasi-experimental, or non-experimental, quasi-experimental, or non-experimental, many researchers develop experimental, many researchers develop their studies to look for differencestheir studies to look for differences they look for differences between or they look for differences between or

among the group or categories of the IV among the group or categories of the IV in relationship to the DVin relationship to the DV

Inferential Statistics

Inferential statistics are used to draw Inferential statistics are used to draw conclusions about a population by conclusions about a population by examining the sampleexamining the sample

POPULATIONPOPULATION

SampleSample

Inferential Statistics

Accuracy of inference depends on Accuracy of inference depends on representativeness of sample from representativeness of sample from populationpopulation

random selection random selection equal chance for anyone to be selected equal chance for anyone to be selected

makes sample more representativemakes sample more representative

Inferential Statistics

Inferential statistics help researchers test Inferential statistics help researchers test hypotheses and answer research questions, hypotheses and answer research questions, and derive meaning from the resultsand derive meaning from the results a result found to be statistically a result found to be statistically

significant by testing the sample is significant by testing the sample is assumed to also hold for the population assumed to also hold for the population from which the sample was drawnfrom which the sample was drawn

the ability to make such an inference is the ability to make such an inference is based on the principle of probabilitybased on the principle of probability

Inferential Statistics

Researchers set the significance level for Researchers set the significance level for each statistical test they conducteach statistical test they conduct by using probability theory as a basis for by using probability theory as a basis for

their tests, researchers can assess how their tests, researchers can assess how likely it is that the difference they find is likely it is that the difference they find is real and not due to chancereal and not due to chance

Alternative and Null Hypotheses

Inferential statistics test the likelihood that Inferential statistics test the likelihood that the alternative (research) hypothesis (Hthe alternative (research) hypothesis (H11) is ) is

true and the null hypothesis (Htrue and the null hypothesis (H00) is not) is not

in testing differences, the Hin testing differences, the H11 would predict would predict

that differences would be found, while the that differences would be found, while the HH00 would predict no differences would predict no differences

by setting the significance level (generally by setting the significance level (generally at .05), the researcher has a criterion for at .05), the researcher has a criterion for making this decisionmaking this decision

Alternative and Null Hypotheses

If the .05 level is achieved (p is equal to or If the .05 level is achieved (p is equal to or less than .05), then a researcher rejects the less than .05), then a researcher rejects the HH0 0 and accepts the Hand accepts the H11

If the the .05 significance level is not If the the .05 significance level is not achieved, then the Hachieved, then the H00 is retained is retained

Degrees of Freedom Degrees of freedom (Degrees of freedom (dfdf) are the way in which ) are the way in which

the scientific tradition accounts for variation the scientific tradition accounts for variation due to errordue to error it specifies how many values vary within a it specifies how many values vary within a

statistical teststatistical test scientists recognize that collecting data can never be scientists recognize that collecting data can never be

error-freeerror-free each piece of data collected can vary, or carry error each piece of data collected can vary, or carry error

that we cannot account forthat we cannot account for by including by including dfdf in statistical computations, scientists in statistical computations, scientists

help account for this errorhelp account for this error there are clear rules for how to calculate there are clear rules for how to calculate dfdf for each for each

statistical teststatistical test

Inferential Statistics: 5 Steps To determine if SAMPLE means come To determine if SAMPLE means come

from same population, use 5 steps with from same population, use 5 steps with inferential statisticsinferential statistics1. 1. State HypothesisState Hypothesis

HHoo: no difference between 2 means; any : no difference between 2 means; any difference found is due to sampling errordifference found is due to sampling error

• any significant difference found is not a TRUE any significant difference found is not a TRUE difference, but CHANCE due to sampling errordifference, but CHANCE due to sampling error

results stated in terms of results stated in terms of probabilityprobability that H that Ho o

is falseis false• findings are stronger if can reject Hfindings are stronger if can reject Hoo

• therefore, need to specify Htherefore, need to specify Hoo and Hand H11

Steps in Inferential Statistics

2. 2. Level of SignificanceLevel of Significance Probability that sample means are Probability that sample means are

different enough to reject different enough to reject HHo o (.05 or .01)(.05 or .01) level of probability or level of confidencelevel of probability or level of confidence

Steps in Inferential Statistics

3. 3. Computing Calculated ValueComputing Calculated Value Use statistical test to derive some calculated value Use statistical test to derive some calculated value

(e.g., t value or F value)(e.g., t value or F value)

4. 4. Obtain Critical ValueObtain Critical Value a criterion used based on a criterion used based on dfdf and alpha level (.05 and alpha level (.05

or .01) is compared to the calculated value to or .01) is compared to the calculated value to determine if findings are significant and therefore determine if findings are significant and therefore reject Hreject Hoo

Steps in Inferential Statistics

5. 5. Reject or Fail to Reject Reject or Fail to Reject HHoo

CALCULATED value is compared to the CALCULATED value is compared to the CRITICAL value to determine if the difference CRITICAL value to determine if the difference is significant enough to reject His significant enough to reject Hoo at the at the predetermined level of significancepredetermined level of significance

If CRITICAL value > CALCULATED value If CRITICAL value > CALCULATED value --> fail to reject H--> fail to reject Hoo

If CRITICAL value < CALCULATED value If CRITICAL value < CALCULATED value --> reject H--> reject Hoo

If reject HIf reject Hoo, only , only supportssupports H H11; it does not ; it does not proveprove H H11

Testing Hypothesis

If reject If reject HHoo and conclude groups are really and conclude groups are really

different, it doesn’t mean they’re different for the different, it doesn’t mean they’re different for the reason you hypothesizedreason you hypothesized may be other reasonmay be other reason

Since HSince Hoo testing is based on sample means, not testing is based on sample means, not

population means, there is a possibility of making population means, there is a possibility of making an error or wrong decision in rejecting or failing to an error or wrong decision in rejecting or failing to reject Hreject Hoo

Type I errorType I error Type II errorType II error

Testing Hypothesis

Type I error -- rejecting Type I error -- rejecting HHoo when it was true (it when it was true (it

should have been accepted)should have been accepted) equal to alphaequal to alpha if if = .05, then there’s a 5% chance of Type I = .05, then there’s a 5% chance of Type I

errorerror Type II error -- accepting HType II error -- accepting Hoo when it should have when it should have

been rejectedbeen rejected If increase If increase , you will decrease the chance of , you will decrease the chance of

Type II errorType II error

Identifying the Appropriate Statistical Test of Difference

One variable One-way chi-square

Two variables(1 IV with 2 levels; 1 DV) t-test

Two variables(1 IV with 2+ levels; 1 DV) ANOVA

Three or more variables ANOVA