7/28/2019 Basics of Data Interpretation by aravind

1/2

Basics of Data Interpretation

I. Statistical Decisions: Remember that for official rules of statistical decision making, there are only

two possible outcomes:

1. The statistic is significant (that is, the correlation, t-test, F-ratio, Tukeys HSD test, whatever) meets or

exceeds the criterion for statistical significance at the stated cutoff ( = .05 or = .01), in which case:

you can reject the null hypothesis (reject the hypothesis that there is no effect)

you can assert that there really is an effect, acknowledging the 5% or 1% chance of Type I error. you can interpret the effect, that is, the difference between the means, as being real.

2. Or, the statistic is not significant (that is, the correlation, t-test, F-ratio, HSD value, etc.) fails to meet the

criterion for statistical significance at the stated cutoff of = .05 or = .01, in which case:

you fail to reject null, meaning that the null hypothesis cannot be rejected but that you cannotclaim it has been proven true, either. You just cant be sure.

you can say that you failed to show an effect of the independent variable (IV), but not that youve

proven that the IV truly has no effect. you cannot interpret any difference between the means. Even if the means are numerically

disparate, the lack of statistical significance means that any difference between the means is likely

due to just random noise variation. Therefore, you cannot treat any numeric difference betweenthe means as a sign of anything. Statistically speaking, there is not difference between the means.

II. Interpretation Decisions: Once you establish the statistical outcome (whether it is statistically

significant or not), it is time to apply that information to the overall final interpretation of the data. In thisclass, we have three possible decisions to choose among: Supported, Unsupported, orDisconfirmed.

1. The researchers hypothesis is supported when two conditions have been met: the relevant statistical test is significant at the stated cutoffAND

the effect is in the right direction (meaning that the conditions that were predicted to have the

larger means actually do have the larger means, or, in a correlational study, that if the correlation

was predicted to be positive it actually is positive). Note: It is vital to understand that even when both these conditions are met, the hypothesis is

merely supportedandnot proven correct. This is for two reasons: (1) the result could be a

Type I error with probability of (.05 or .01); and (2) there may be other theories which predict

the same finding, which would also be supported by this finding, so we cant say that this one

theory is proven right!

2. The researchers hypothesis is unsupported whenever the statistical test is not significant (sometimes

people call this a null result). This means that: the null hypothesis (that there is no effect) cannot be rejected, but neither is it proven true. the results of the study are inconclusive; the researchers hypothesis is not supported but it is not

proven wrong either. The absence of evidence does not constitute evidence of absence. Note: Sometimes researchers will want to argue on the basis of a non-significant finding (a null

result) that the null hypothesis is true and should be accepted, and that there really is no effect

of the IV. People sometimes get away with this, but it is commonly viewed as a tough, uphill,

battle to make an argument from null results.

3. The researchers hypothesis is disconfirmed when there is a statistically significant result, but that what

happened across the means is the opposite of, or at least markedly different from, what was supposed to

have happened according to the researchers hypothesis. In this case:

7/28/2019 Basics of Data Interpretation by aravind

2/2

the fact that the result is statistically significant means that there really is an effect present in the

data, meaning that you can reject the null hypothesis. even though the null is rejected, the direction of the effect is different from the researchers

hypothesis, so rejection of null does not necessarily mean support for the researchers hypothesis. you can use the term disconfirmed because a statistically real effect occurred that was

opposite of, or different from, what was supposed to occur. This is different from unsupported

which means just the lack of a definitive answer one way or the other. That is, its the presence

of disconfirming information rather than just the absence of supportive information.

III. An Important Elaboration. In the world of data interpretation, the Disconfirmed conclusion is often

viewed as more decisive than the Supported conclusion. Look at it this way:

1. Supported means that the result is consistentwith what the theory predicted, but other theories might

predict the same thing, or, the theory could be wrong in other respects that werent being tested in this

particular experiment. So, at best, the present study is just one piece of evidence, among many that wouldultimately be needed, to prove the theory right.

whereas:

2. Disconfirmed means that a statistically real effect occurred that was not supposed to happen, was not

predicted, and cannot be explained by the hypothesis/theory. This means that the theory is directly

contradicted and so must be wrong in at least this respect. So the disconfirmed conclusion can prove atheory wrong but the supported conclusion doesnt prove the theory right.

This is why many researchers construct experiments so that a significant result is what a theory says should

not happen. That is, we design experiments to disprove theories rather than to prove them.