Introduction the General Linear Model (GLM) l what “model,” “linear” & “general” mean l bivariate, univariate & multivariate GLModels l kinds of variables

Introduction the General Linear Model (GLM)

what “model,” “linear” & “general” mean bivariate, univariate & multivariate GLModels kinds of variables some common models

“General Linear Model”

• “Model” means that we are usually interested in predicting or “modeling” the values of one variable (criteria) from the values of one or more others (the predictors)

• “Linear” means that the variables will be “linearly transformed (* & /) and “linearly combined” (+ & -) to produce the model’s estimates

• “General” means that the model intends to provide a way to model & test RHs: about any combination of criterion and predictor variables (i.e., any model), and to test RHs: about comparisons among models

Regression “vs.” GLM

The “constant” is often represented differently in GLM than in multiple regression …

Single predictor models

single predictor regression y’ = bx + a

single predictor GLM y’ = b0 + b1x1

Multiple predictor models

multiple predictor regression y’ = b1x1 b2x2+ a

multiple predictor GLM y’ = b0 + b1x1 + b2x2

Common kinds of GLModels• Bivariate one criterion & one predictor

simple regression

y’ = b0 + b1x

• Univariate one criterion & multiple predictors multiple regression in all its forms

y’ = b0 + b1x1 + b2x2 + b3x3

• Multivariate multiple criterion & multiple predictors canonical regression in all its

forms

b0 + b1y1 + b2y2 = b0 + b1x1 + b2x2 + b3x3

Common kinds of variables

Quantitative variables

• Raw variable

• Centered variables X – mean

• Mean 0 simplifies math of more complicated models

• Re-centered variables X a more meaningful value

• Change “start” or “stop” values

• E.g., “aging & intellectual decline”

• Mathematical “trick” to get the desired model/weights• selecting which group or value will be represented in model’s bs


Quadratic quantitative variables

• X2 – actually represents combination of linear + quadratic

• Xcen2 – represents the “pure” quadratic term

• Model with X2 will have ≈ R2 as model with Xcen + Xcen2

• A model with a quadratic term should always include the linear term for that variable


2-group variables

• Unit coding (usually 1-2)

• Dummy Coding• “control” or “comparison” group coded 0

• “treatment” or “target” group coded 1

• Effect Coding• “control” or “comparison” group coded -1

• “treatment” or “target” group coded 1


k-group variables

• Raw coding (usually 1-2-3, etc.)

• Dummy Coding• “control” or “comparison” group coded 0• “treatment” or “target” groups coded 1 on one

variable & 0 on all others• the full set of codes must be included in the model

• Effect Coding• “control” or “comparison” group coded -1• “treatment” or “target” group coded 1 on one

variable & 0 on all others• the full set of codes must be included in the model


K-groups variables, cont.

• Comparison coding• Combining simple and complex analytical comparison codes

to represent specific, hypothesis driven, group comparisons

• E.g., Say you have 4 groups and RH: that…• Group 1 has higher scores that the average scores of groups 2-4

the codes would be gp1 = 3 gp2 = -1 gp3 = -1 gp4 = -1

• Groups 2 & 3 have higher average scores than do 1 & 4 the codes would be gp1 = -1 gp2 = 1 gp3 = 1 gp4 = -1

• Group 2 has higher scores than the average scores of groups 3-4 the codes would be gp1 = 0 gp2 = 2 gp3 = -1 gp4 =

-1

• Usually havea set of k-1 codes


K-groups variables, cont.

Polynomial coding If the groups represent a quantitative continuum, you

use codes to represent different polynomial functions (linear, quadratic, cubic, etc.) to explore the shape of the relationship between that variable and the criterion

E.g., for a 5-group variable, the polynomial codes are …• Linear -2 -1 0 1 2• Quadratic 2 -1 -2 -1 2• Cubic -1 2 0 -2 1• Quartic 1 -4 6 -4 1• the full set of codes must be included in the model


Ordered-category variablesSometimes you have a quantitative variable that you want to

change into a set of ordered categoriese.g. % grade into “A” “B” “C” “D” “F”e.g. % grade into “Pass” “Fail”

e.g. aptitude test scores into “remedial” “normal” “gifted”

Sometimes this is done to help with “ill-behaved distributions” e.g. frequency variable with mean=1.1, std=8.4, sk=4.2 e.g. frequency variable with 60% “0” 38% “1” max =

118

Important because skewed univariate distributions can “create” apparently nonlinear bivariate relationships


Ordered-category variables, cont.Once you form the ordered categories (using “IF,”

“RECODE” or other transformations), you can enter those variables into the GLM in different ways

• Using the category values (e.g., 1, 2, 3, etc) ***• Centering or re-centering the category values ***• Dummy codes of the category values• Effect codes of the category values• Polynomial codes of the category values***

*** indicates approaches that make assumptions about the interval nature of the variable and/or its normal distribution, with which not everyone agrees!


Interactions• Interactions represent the “joint effect” or “non-additive

combination” of 2 or more predictors as they relate to a criterion (or set of criteria in the multivariate case).

• They are the “moderation,” “it depends,” “sometimes,” or “maybe” that makes our science and statistical analyses so interesting.

• Interactions can be formed from the combination of any 2 or more variables of the types just discussed.

• There are some “guidelines” about forming, including and interpreting interaction terms.


Interaction “Guidelines”• When including a 2-way interaction, both related main

effects must be included

• When including a 3-way interaction, all 3 main effects and all 3 2-way interactions must be included

• When including a non-linear interaction term, the related linear and nonlinear main effects, and linear interaction terms must be included

• The associated terms can not exceed the df of the variables involved (except for quantitative variables)

Common kinds of GLModels

“Linear” Multiple regression models

y’ = b0 + b1x1 + b2x2 + b3x3

Can include any of the variable types:

• Quantitative (raw, centered or re-centered)

• 2- or k-group (with dummy, effect, or comparison coding)

• Ordered category (coded)


“Non-Linear” Multiple regression models with quant variables

y’ = b0 + b1x1 + b2x12 + b3x2 + + b4x2

2


• Linear terms should be centered

• Non-linear terms should be centered then powered

• Non-linear terms above quadratic should be based on theory

• Include linear term for all non-linear terms, at least at first


“2-way Interaction” Multiple regression models

y’ = b0 + b1x + b2z + b3xz


• Quantitative variables should be centered

• 2- or k-group variables should be coded

• Interaction terms formed as product of main effect terms

• Must included main effects terms for any interaction variable


“3-way Interaction” Multiple regression models

y’ = b0 + b1x + b2z + b3v + b4xz + b5xv + b6zv +

b7xzv


• Quantitative variables should be centered

• 2- or k-group variables should be coded

• Interaction terms formed as product of main effect terms

• Must included main effects terms for any interaction variable


2-group ANOVA models

y’ = b0 + b1x1

“X” is a dummy or effect coded 2-group variable



y’ = b0 + b1x1 + b2x2

“X1” & “X2” are a dummy or effect codes for a 3-group

variable



y’ = b0 + b1x1 + b2x2 + b3x3

“X1,” “X2” & “X3” are a dummy or effect codes for a

4-group variable


2x2 Factorial ANOVA model

y’ = b0 + b1x + b2z + b3xz

“X” is a dummy or effect code of 1st 2-group variable“Z” is a dummy or effect code of 2nd 2-group variable“XZ” represents the interaction of “X” and “Z”


2x3 Factorial ANOVA model

y’ = b0 + b1x1 + b2z1 + b3z2 + b4xz1 +

b5xz2

“X1” is a dummy or effect code of 1st 2-group variable“Z1” & “Z2” are dummy or effect codes of 2nd k-group variable“XZ1” & “ZX2” represent the interaction of “X” and “Z”


2-group ANCOVA models

y’ = b0 + b1x + b2z


“Z” is the covariate (dummy coded or quantitative)


2-group ANCOVA models with covariate interaction

y’ = b0 + b1x + b2z + b3xz



“XZ” represents the interaction of “X” and “Z”


3-group ANCOVA models

y’ = b0 + b1x1 + b2x2 + b3z


variable



3-group ANCOVA model with covariate interaction

y’ = b0 + b1x1 + b2x2 + b3z + b4xz1 + b5xz2


variable

“Z” is the covariate (dummy coded or quantitative) “XZ1” & “ZX2” represent the interaction of “X” and “Z”


2x2 Factorial ANCOVA model

y’ = b0 + b1x + b2z + b3xz + b4v“X” is a dummy or effect code of 1st 2-group variable“Z” is a dummy or effect code of 2nd 2-group variable“XZ” represents the interaction of “X” and “Z”“V” represents the covariate


2x2 Factorial ANCOVA model with covariate interactions

y’ = b0 + b1x + b2z + b3xz + b4v + b5xv + b6zv +

b7xzv “X” is a dummy or effect code of 1st 2-group variable“Z” is a dummy or effect code of 2nd 2-group variable“XZ” represents the interaction of “X” and “Z”“V” represents the covariate“XV” represents the interaction of “X” and “V”“ZV” represents the interaction of “Z” and “V” “XZV” represents the interaction of “X,” “Z” and “V”

Documents

Introduction the General Linear Model (GLM) l what “model,” “linear” & “general” mean l bivariate, univariate & multivariate GLModels l kinds of variables