1

Hypothesis Testing and P ValueBY DR ZAHID KHAN

SENIOR LECTURER KING FAISAL UNIVERSITY,

KSA

2Two ways to learn about a population

Confidence intervals Hypothesis testing

3HYPOTHESIS

What do you mean by a Hypothesis?

A hypothesis is a proposition that is –

assumed as a premise in an argument / claim

set forth as an explanation for the occurrence of some

specified group of phenomena

A hypothesis is a prediction about the outcome of an

experiment. In market research this could be the result of

the out come of a focus or field study

4Why do we make hypotheses?

The practice of science traditionally involves formulating and testing hypotheses

Hypotheses are assertions that are capable of being proven false using a test of observed data

Hypothesis testing is a procedure through which sample data is used to evaluate the credibility of a hypothesis

Hypothesis is basically used to check the estimate whether its purely by chance or its real and not due to chance. ( Null & Alternative Hypothesis).

5

Null Hypothesis The null hypothesis typically corresponds to a general

or default position Making this assertion will make no difference and

hence cannot be proven positively

Alternate Hypothesis An alternate hypothesis asserts a rival relationship

between the phenomena measured by the null hypothesis

It need not be a logical negation of the null hypothesis as it only helps in rejecting or not rejecting the null hypothesis

TYPES OF HYPOTHESIS

6

Customers going to shops playing music spend more.

Independent Variable: Music in the shops

Dependent Variable: Amount spent in shops

Dependant and independent variables

7Null & Alternative Hypothesis

Null Hypothesis.

There is no difference between spending of customers in music playing shops and music playing shops and any difference is by chance.

Alternative Hypothesis.

The difference between customers spending in music playing shops is higher than non-music playing shops and the difference is real, its not by chance.

8

1. Obtain a random sample of customers who go to shops with music

2. Check shop spending

3. Compare sample data to hypothesis

4. Make decision:1. Reject the null hypothesis based on the difference

between the two groups with P value less than 0.05.

2. Fail to reject the Null Hypothesis due to a difference of greater than 0.05 so alternative hypothesis is rejected.

Example -- Continued

9

What are errors in Hypothesis Testing?

The purpose of Hypothesis Testing is to reject or not reject the Null Hypothesis based on statistical evidence

Hypothesis Testing is said to have resulted in an error when the decision regarding treatment of the Null Hypothesis is wrong

TYPES OF ERRORS

10

TYPES OF ERRORS

Actual State of Affairs

Belief Decision H0 is False H0 is True

H0 is False Reject H0 Correct Rejection1 - Power

Type I Error

H0 is True Fail to Reject H0 Type II Error

Correct Failure to Reject1 -

11

During the Hypothesis Testing,α – is the probability of occurrence of a Type-I Error

β – is the probability of occurrence of a Type-II Error

Relationship between α and β For a fixed sample size, the lower we set value of α,

the higher is the value of β and vice-versa In many cases, it is difficult or almost impossible to

calculate the value of β and hence we usually set only α

α, β AND THE INTER-RELATIONSHIP

12

Jury’s Decision

Did Not Commit Crime Committed Crime

Guilty Type I ErrorConvict Innocent Person

Correct VerdictConvict Guilty Person

Not Guilty Correct AcquittalFail to Convict Innocent Person

Type II ErrorFail to Convict Guilty Person

13

1. Alpha: probability of committing a Type I error1. Reject H0 although it is true

2. Symbolized by 3. In research or type 1 error basically kills patients

because a useless drug is going to be prescribed.

4. Type 2 error kills researcher because a valid and significant result is not published.

2. Obtained result attributed to:1. Real effect (reject H0)

2. Chance ( Accept Null)

Level of Significance

14

1. P-value is a probability statement:

2. Probability that the test will correctly reject a false null hypothesis.

3. A p-value is based on chance variation: how likely is this sample or a more extreme one as a result of chance alone?

4. A p-value is the probability that the null hypothesis is true

5. A p-value is the probability that the null hypothesis is false.

Statistical Power

15

Interpreting the weight of evidence against the Null Hypothesis for rejecting / not rejecting Ho

If the p-value for testing Ho is less than –

< 0.05, we have strong evidence that Ho is false

< 0.01, we have very strong evidence that Ho is false

P value is taken as 0.05 or 5% because it is a standard icon & it

nearly corresponds to the difference of two standard errors.

INTERPRETING RESULTS

16Decision Rule used with P-Value to Make Conclusions

A p-value measures the strength of evidence against an hypothesis

if a p-value is small, then either the null hypothesis is false or we got a very unlikely sample

small p-values lead us to reject null hypotheses, and support alternative hypotheses

The smaller the p-value is, the more convincing the evidence is against the null hypothesis

if a p-value is large, the null hypothesis could still be false, but n may be too small to reject the null.

17One Sided & Two Sided Tests

Consider two means A & B.

One sided test only tells you that A > B.

Two sided tests tells you that either A>B or A <B so leaving you with two options.

Mostly Two sided tests are used except in cases of equivalence tests like Lumpectomy done for Breast surgery as well as radical Mastectomy.

One sided test would be whether Lumpectomy is worst for survival than Radical Mastectomy and we don't bother about better survival results.

One sided tests can also be applied for variables which will only go in one direction like Height, Age etc.

18Relationships between confidence interval, alpha & P Values.

p-values can be referenced to alpha, or the chance of making a type one error

a 95% confidence interval will reject any value of the null hypothesis that is outside the interval at a 5% significance level for a two-sided test.

A larger sample size with the same sample mean will result in a smaller p-value (i.e., stronger evidence)

19Cautions and Limitations p-values are about data, not parameters A null hypothesis is right or wrong, one or the other A p-value indicates that there may be an effect and how strong

the evidence is that there is an effect. It does not tell how large an effect there is.

A p-value is only as good as your data. The error involved may be due to sampling.

large samples can lead to small p-values without resulting in practical significance (e.g., statistical significance does not imply practical significance)

Interpreting a p-value (or its implication) is relative to the context or consequence of the problem (and data).

There is nothing sacred about .05

20Any Questions !!!!

Thank You.