Some basic statistical tests & more on basic statistical analysis Communication Research Week 11 with help from: Carey, J & Dimmitt, C. (2003) Statistical

Some basic statistical tests & more on basic statistical analysis

Communication Research

Week 11with help from:

Carey, J & Dimmitt, C. (2003) Statistical Analysis: Is Change Real? www.umass.edu/schoolcounseling/ WelcometoAmherstMassachusetts/StatisticalAnalysis.ppt [accessed 10 Oct 2006]

http://www.statsoft.com/textbook/stathome.html

Communication Research 2

Why Statistical Analysis? After we gather and compute our data, we want to be

sure that the scores of two groups really are different. We want to be sure that the differences we see are

not just due to chance. If we are basing decisions on real differences our

behavior is directed and purposeful. If we are basing decisions on differences that are

only due to chance our behavior is random and chaotic.


Statistical Tests

Allow us to estimate the likelihood that the apparent differences between groups are real and not due to chance.

These tests have the built in capacity to take the number of people per group and the variability of the data into account when making these estimates.


Measuring Variables Variables – the things we measure, control or manipulate Independent variables (IV) are usually those that are

manipulated Dependent variables (DV) are only measured or registered They differ in how well they can be measured and the type of

measurement scale used Two or more variables are related if, in a sample of

observations, the values systematically correspond to each other for these observations

eg height is considered related to weight because typically tall people are heavier than short ones; IQ is related to the number of errors in a test, if people with higher IQs make fewer errors


Why are relations between variables considered important?

The philosophy of science believes that there is no other way of representing “meaning” except in terms of relations between some quantities or qualities

Statistical significance (p-value) of a result is the probability that the observed relationship (eg between the variables) or a difference (eg between the means) in a sample occurred by pure chance (“luck of the draw”) and that in the populations from which the sample was drawn, no such relationship or differences exist

In other words, the statistical significance of a result tells us something about the degree to which the result is “true” (ie representative of the population)


Example – "Baby boys to baby girls ratio."

Consider the following example from research on statistical reasoning (Nisbett, et al., 1987). There are two hospitals: in the first one, 120 babies are born every day, in the other, only 12. On average, the ratio of baby boys to baby girls born every day in each hospital is 50/50. However, one day, in one of those hospitals twice as many baby girls were born as baby boys. In which hospital was it more likely to happen?

The answer is obvious for a statistician, but as research shows, not so obvious for a lay person: It is much more likely to happen in the small hospital.

The reason for this is that technically speaking, the probability of a random deviation of a particular size (from the population mean), decreases with the increase in the sample size.


Data characteristics that help determine the statistical test used

Type of data used – nominal, ordinal, interval, ratio

Two groups vs more than two groups Whether groups are matched (“paired”) or

unmatched Whether groups are small or large Whether the data are normally distributed

(continuous data)


Different data/variable typesData type Description Example

Nominal

Allow for only qualitative classification – they can be measured only in terms of whether the individual items belong to some distinctively different categories, but we cannot quantify or even rank order those categories.

For example, all we can say is that two (2) individuals are different in terms of variable A (eg of a different race), but we cannot say which one "has more" of the quality represented by the variable. Typical examples of nominal variables are gender, race, color, city, etc

Ordinal

Allow us to rank order the items we measure in terms of which has less and which has more of the quality represented by the variable, but still they do not allow us to say "how much more."

Eg socioeconomic status of families. For example, we know that upper-middle is higher than middle but we cannot say that it is, for example, 18% higher. Also this very distinction between nominal, ordinal, and interval scales itself represents a good example of an ordinal variable. For example, we can say that nominal measurement provides less information than ordinal measurement, but we cannot say "how much less" or how this difference compares to the difference between ordinal and interval scales.


Different data/variable types

Data type Description Example

Interval

Allow us not only to rank order the items that are measured, but also to quantify and compare the sizes of differences between them. For example, temperature, as measured in degrees Fahrenheit or Celsius, constitutes an interval scale.

Eg We can say that a temperature of 40 degrees is higher than a temperature of 30 degrees, and that an increase from 20 to 40 degrees is twice as much as an increase from 30 to 40 degrees

Ratio

Are very similar to interval variables; in addition to all the properties of interval variables, they feature an identifiable absolute zero point, thus they allow for statements such as x is two times more than y.

Typical examples of ratio scales are measures of time or space. For example, as the Kelvin temperature scale is a ratio scale, not only can we say that a temperature of 200 degrees is higher than one of 100 degrees, we can correctly state that it is twice as high. Interval scales do not have the ratio property. Most statistical data analysis procedures do not distinguish between the interval and ratio properties of the measurement scales.


Example 2: After implementation of a family math education intervention, Latino/a students average 4th Grade MCAS scaled score increased from 206 to 215.

Hypothesis (Ha). The two groups are really different.

Null Hypotheses (Ho). The two groups are not different, the apparent difference is due to chance.


Example 2: After implementation of a family math education intervention, Latino/a Students’ average 4th Grade MCAS score increased from 206 to 215.

The variability of the outcome data is a major factor in determining whether the differences are real or due to chance.

At the TAB, in a straight bet, how much would you be willing to wager the Ha is true if you knew that, if students retake the MCAS within a month

90% of the time their two scores differ by less than 2 points.

90% of the time their two scores differ by less that 10 points.

90% of the time their two scores differ by less than 50 points.


Example 1: 70% of White and 40% of African American 3rd graders score Advanced or Proficient on the MCAS Reading Test.

The number of people in two groups is a major factor in determining whether differences are real or due to chance.

At the TAB, in a straight bet, how much would you be willing to wager the hypothesis is true if you knew that:

The percentages are based on 10 students from each group.




Parametric vs non parametric tests

Since we have two types of data we need two types of statistical tests.

Parametric – the DV is a continous variable (eg age in years) so it makes sense to calculate the mean and SD

Non parametric – the DV is a count (nominal data) or a ranking (ordinal data) and so it makes no sense to measure a means eg “the average gender of Australians is 1.5”

Parametric Tests are generally more powerful, meaning that if there is a real difference between the groups its easier to find it with a Parametric Test


Examples of parametric tests Independent t-test or a comparison of two means Looks for a difference between two groups (eg men and

women) on a particular variable (eg whether they kiss on the first date)

Paired t-test eg such as how someone feels about drink driving before they get caught, and how they feel about it afterwards.

In an SPSS output table, the Sig (2-tailed) value is the significance value – the likelihood that the result could happen by pure chance.

If the value is less than 0.05, the chance is less than 5%, so the significance of the difference is 95% – this is therefore highly significant


Choosing a significance level Statistical Tests do not give us information that allows

us to definitively say whether an observed difference between groups is real or just due to chance.

Statistical Tests do give us an estimate of the likelihood that observed difference between groups results from chance.

We must decide what criteria we will use for deciding whether a difference is real.


Choosing a Significance Level

We do this by choosing a Significance Level .25 25% chance difference is due to chance .10 10% chance difference is due to chance .05 5% chance difference is due to chance .01 1% chance difference is due to chance


T-Test in SPSS

SPSS will allow you to do all of these tests quite easily


If the value is less than 0.05, the chance is less than 5%,

so the significance of the difference is 95% which is

highly significant.

Levene’s test, checks to see whether the variances of the two variables are relatively similar. If the significance for Levene's test is 0.05 or below, then the

‘Equal Variances Not Assumed’ t-test result (the one on the bottom) is

used. Otherwise you use the ‘Equal Variances Assumed’ test

(the one on the top)


T-Test for Independent Samples

Remember, we need to know two other things in order to ascertain the likelihood of chance creating this size of a difference: The number of people in each group The variability of the scores

The number of people is easy, and is counted in the frequency table (n)


Variability

In order to know whether a difference between two means is important, we need to know how much the scores vary around the means.


Variability

Holding the difference between the means constant

With High Variability the two groups nearly overlap

With Low Variability the two groups show very little overlap


Measuring Variability

Medium Variance

High Variance

Low Variance


Measuring Variability

Usually it’s easier to work with the square root of the variance.

This statistic is called the Standard Deviation. SPSS statistical tests will calculate the SD for you


ANOVA ANOVA is an acronym; ANalysis Of VAriance. It is an extension of the two-tailed t-test, and is generally used to test for

significant differences between means. The name is derived from the fact that in order to test for statistical

significance between means, we actually compare (or analyse) variances. For two-group comparisons, ANOVA will give results identical to a t-test,

but when the design is more complex, ANOVA offers numerous advantages that t-tests cannot provide (even if you run a series of t- tests comparing various cells of the design).

For example, it often happens in research practice that you need to compare more than two groups (e.g., drug 1, drug 2, and placebo), or compare groups created by more than one independent variable while controlling for the separate influence of each of them (such as Gender, Type of Drug, and Size of Dose).





Independent T test

John Robert


Dependent T-test

John John


Oneway ANOVA

John Robert Kevin Tom


Repeated Measures ANOVA

John John John John


Factorial ANOVA

John Robert Kevin Tom

Janine Roberta Katie Teresa


Mixed ANOVA

John John John John

Janine Janine Janine Janine


Chi2 Test

Chi Square (X2) - is a non-parametric test It uses nominal data and checks to see if there are

significant differences between/among groups compared to what would be expected.

The crosstabulation table tells you whether selected variables are related to other selected variables; the chi-square table tells you what the degree of certainty is.

Chi-square is based on the fact that for a two-way table, we can compute the frequencies that we would expect if there was no relationship between the variables.


Chi2 Example

Suppose we ask 20 men and 20 women to choose between two brands of soft drink - brands A and B. If there is no relationship between preference and gender, then we would expect about an equal number of choices of brand A and brand B for each gender.

The chi-square test becomes increasingly significant as the numbers deviate further from this expected pattern; that is, the more this pattern of choices for men and women differs.






If our Chi2 test statistic has exceeded the critical value for:

The .10 significance level it would mean that there was only a 10% chance of seeing a difference that large that resulted from chance.


The .025 significance level it would mean that there was only a 2.5% chance of seeing a difference that large that resulted from chance.


If our Chi2 test statistic has exceeded the critical value for:


The .005 significance level it would mean that there was only a 0.5% chance of seeing a difference that large that resulted from chance.


Pearson correlation The Pearson correlation looks for a relationship between

two variables and generates a mathematical index of the relationship between them.

The value lies between -1.00 and +1.00, and the bigger the number the stronger the relationship.

Pearson correlation indicates trends - one thing increases (or decreases) as another thing increases (or decreases).

A negative value indicates that low scores on one variable go with high scores on the other variable, while a positive value indicates that a high score of one variable goes with a high score on the other variable.



Note that SPSS will allow you to calculate different tests of significance as well as note significant correlations


Documents

Some basic statistical tests & more on basic statistical analysis Communication Research Week 11 with help from: Carey, J & Dimmitt, C. (2003) Statistical