PRINCIPLES OF HYPOTHESIS TESTING

1

PRINCIPLES OFPRINCIPLES OF

HYPOTHESIS TESTINGHYPOTHESIS TESTING

2

A Quick Review of Important Issues About Sampling:

• To examine the sample’s attributes (sample statistics) as ESTIMATES of the population’s characteristics (population parameters)

use sample characteristics to make inferences about the population.

• Estimating, by definition, involves some error (i.e., sampling error/bias

Resulting from the fact that the sample may not mirror characteristics of the population).

Why Sampling?

3

HYPOTHESIS TESTING OFTEN INVOLVES:

a. comparing groups regarding differences in means or proportions, or

b. Examining strength and direction of relationships between two variables

Common Types of Research Hypotheses and the Related Common Types of Research Hypotheses and the Related Statistical Data Analysis Methods:Statistical Data Analysis Methods:

a.Checking for Presence/Absence of Relationship(s) Among Variables (and direction/strength of the relationship)

a.Bivariate (e.g., Pearson Correlation—r)a.Between one variable and another: Y = a + b1 x1

• Multivariate (e.g., Multiple Regression Analysis)Between one dep. var. and an independent variable,while holding all other independent variables constant:

Y = a + b1 x1 + b2 x2 + b3 x3 + … + bk xk

– Checking for Presence/Absence of Difference(s) Among Groups

a.Difference(s) in Proportions (Chi Square Test—2)

b.Difference(s) in Means (Analysis of Variance)

NOTE: Statistical tests of hypotheses always report the result of testing the null hypothesis.

The researcher will then have to restate the results in terms of finding/not finding support for the original research hypothesis.

4


• If we reject the null, the conclusion is that we have found a statistically significant relationship/difference.

– When we don’t reject the null, we infer that . . .?“…any difference/relationship that may be apparent from sample data is likely to be the result of . . .??

QUESTIOQUESTION: When testing hypotheses regarding presence of a relationship/ difference, what does “NULL HYPOTHESIS” (H0) refer to?

sampling error (i.e., an artifact of the particular sampleartifact of the particular sample being used)”.

Null Hypothesis in most cases states:“There is no relationship or no difference.”

5

A Quick Review of Important

Issues About Sampling: • So, when using sample data to test hypotheses

and make judgments about the population, there is always a chance for reaching erroneous conclusions about the population.

• What do we mean by What do we mean by erroneous conclusionserroneous conclusions??• What types of erroneous conclusions can we reach when testing What types of erroneous conclusions can we reach when testing

hypotheses?hypotheses?

6

Important Notes About Sampling

Two possible types of erroneous conclusions from sampling error, when testing hypotheses:

TYPE I ERROR and TYPE II ERROR

7

Important Notes About Sampling Type I Error?Rejecting a true “null hypothesis” (erroneously)

• Rejecting the null when we should not (i.e., when the null is true)

Null Hypothesis?States States ““There is NO relationship, there is NO difference, etc.There is NO relationship, there is NO difference, etc.””

““Rejecting the nullRejecting the null”” refers to concluding…? refers to concluding…?

Concluding that: Concluding that: ““There is a significant relationship/ differenceThere is a significant relationship/ difference””

So, type I error (“Rejecting a True Null”) means?• No relationship/difference exists, but from sample evidence

we come to the conclusion that a significant relationship/ difference does exist.

• Example: A drug is really not effective, but we conclude it is.

• Conclusion: Type I Error involves finding “something” that does not really exist—i.e., a case of “False positive” 8


Sample

9

Important Notes About Sampling Type II Error?

• Accepting a false null hypothesis (or failing to reject a false null hypothesis)

False Null means?• “A relationship/difference does in fact exist”.

So, “accepting a false null” (i.e., type II error) means?• A relationship/difference does in fact exist, but from

sample evidence we fail to detect it (fail to reject the null).

– Come to the conclusion that there is no relationship/difference (in the population).

• Example; A drug is really effective, but our study shows it is not.

10

Important Notes About Sampling Type II Error:

CONCLUSION: Type II Error represents failing to find “something” that does exist; it represents a case of “False Negative.”

Sample

11

A Quick Review of Important

Issues About Sampling:

– Statistical tests of significance assess the likelihood of reaching an erroneous conclusion when using sample data. In fact, they always assess the likelihood of type I error.

• They assess the probability that the relationship/ difference we have found (using sample data) may simply be an artifact of the particular sample we have happened to end up with (i.e., is caused by sampling error).

– In fact, when using data from the entire population (e.g., a census):• No chance of sampling error exists• No need for conducting statistical tests of significance.

When testing hypotheses, what is the purpose of statistical testing (significance testing)?

12

HYPOTHESISHYPOTHESIS TESTINGTESTING

• Statistical tests of significance alway assessthe likelihood/probability of type I error ( )when using sample data.

• Once a test is conducted and is determined, we willhave to decide if we are able/willing to tolerate the riskinvolved in rejecting the null (and, thereby, to report what…?)

… that the relationship/difference detected (from sample data), is too large to be attributed to chance/sampling error.

That is, decide whether we should consider the relationship/difference “statistically significant.”

Sample

13


The complement of :• 1- or confidence level.

What does 1- represent?probability of accepting (not rejecting) the null when it

is true:– concluding NO relationship/difference exists, when

indeed it DOES NOT exist (i.e., chance of not finding what does not exist)—a correct conclusion

The probability of committing Type I Error is called:

(alpha) or significance level.

15


• It requires us to:a)a) DetermineDetermine the likelihood of null being true the likelihood of null being true (( and,

thus, the riskrisk (of being wrong) that we would be we would be takingtaking if we decide to to reject the nullreject the null (, and

b)b) Decide Decide whetherwhether we are willing/able to toleratewilling/able to tolerate that risk (level) by actually rejecting the null…

c) and reporting that we have found a“significant” relationship/difference.

a)a) So, generally speaking, So, generally speaking, when should when should we bewe betempted to tempted to reject the nullreject the null?? When (is ___large or when it is ___small?

So, So, testing the plausibilitytesting the plausibility of hypothesis/propositions of hypothesis/propositions

(i.e., decision to reject/not reject H (i.e., decision to reject/not reject H00) is a ) is a probabilistic decisionprobabilistic decision...... ??

16

HYPOTHESISHYPOTHESIS TESTINGTESTING

A small means that…

•…the CHANCECHANCE of NULL NULL being TRUETRUE is TOO SMALL TOO SMALL to warrant ACCEPTING ACCEPTING it .

•…if we decide to reject the null (i.e., conclude that we have found a relationship/difference), we stand a relatively small chance of being wrong.

• …rejecting the null is a relatively safe bet.

•…the difference/relationship found is statistically significant .

NOTE:NOTE:

Small Small rejecting the null rejecting the null finding a finding a statistically significantstatistically significant relationship/difference reporting that the relationship/difference reporting that the relationship/differencerelationship/differencefound (from sample evidence) is too large to be attributed to chance/found (from sample evidence) is too large to be attributed to chance/sampling errorsampling error

17


BUT HOW DO YOU MEASURE BUT HOW DO YOU MEASURE ??

How would you determine what the actual level is (i.e., how much risk of being wrong you would actually be taking if you were to decide to reject the null?

ANSWER…(a)Look up the actual from a table of probability distribution

for the test statistic being used,OR(b) More conveniently, rely on your statistical software (e.g.,

SPSS) to compute and report the actual “Sig.” or “Prob.”) level for you.

18


DECIDING ON AN DECIDING ON AN that I CAN TOLERATE!!!that I CAN TOLERATE!!!

• BUT, WAIT ... How am I supposed to know what odds of

being wrong I should be willing/able to tolerate as I consider rejecting the null?

A SIMPLE ANSWER:

5% is conventionally considered to be a reasonable/small enough risk to be tolerable in most situations.

19


• IS THERE A RULE OF THUMB TO FOLLOW WHEN TESTING

HYPOTHESIS? YES!• WHAT IS IT?

THE GOLDEN RULE: THE GOLDEN RULE: When testing a hypothesis,

•if the reported (e.g., “sig.” in SPSS) turns out tobe less than or equal to 0.05, reject the null and report a statistically significant relationship/difference Because the odds of being wrong would be tolerable).

•Otherwise, refrain from rejecting the null, on the grounds that the odds of committing an error (i.e., rejecting a true null) would be prohibitive.

• And, as a result, report . . . ?

20


EXCEPTIONS TO THE RULE?Use a smaller (e.g., < 0.01 ) when:

1. Sample size is relatively large2. Consequence of committing type I error is

serious/costly (i.e., False positive results are very costly)• (e.g., H0: Capital punishment is not a strong

deterrent for criminal behavior.)

Use a larger (e.g., < 0.10 ) when:• Sample size is relatively small• Conducting exploratory research whose results provide

the basis for further research

21


QUESTIONS OR COMMENTS

?

Documents

PRINCIPLES OF HYPOTHESIS TESTING