Teaching Styles Revisited 30 Years of Modelling · 2010. 4. 13. · Teaching Styles Revisited | 30 Years of Modelling John Hinde Science Foundation Ireland Funded Bio-SI Project &

Teaching Styles Revisited — 30 Years of Modelling

John Hinde

Science Foundation Ireland Funded Bio-SI Project & Statistics Group,School of Mathematics, Statistics and Applied Mathematics

National University of Ireland, GalwayIreland

[email protected]

Murray Aitkin Meeting, RSS, London

15 April 2010

John Hinde (NUIG) 15 April 2010 1 / 24

Summary

1 Teaching Styles Study

2 Teaching Styles Study ReanalysisClustering Teachers — Latent Class ModelTeaching Style & Pupil ProgressPupil Personality Clustering

3 Teaching Styles Analyses Today

4 Murray’s Legacy


Teaching Styles Study (Bennet, 1976)

Study of relationship between Teaching Style and Pupil Performance inprimary schools

Questionnaire on 28 items (in 6 areas) of classroom behaviour → 36binary items

sent to all 871 primary in Lancs & Cumbria for 3rd and 4th yearteachers88% response ratesubsequent analysis based 468 4th year teachers

Principal component analysis & cluster analysis

12 cluster solution chosen78 teachers not clearly classified into any clusterclusters ordered from extremely formal to extremely informal


Teaching Styles Study (Bennet, 1976)

37 teachers selected to represent 7 clusters (2 informal, 3 mixed, 2formal)

950 Pupils in classes of selected teachers followed for one year

pre-tested on entry to class — reading, mathematics, Englishpost-tested prior to exit from schoolpersonality tests — 15 personality variables

Analysis of Covariance on post-test scores

used to test differences between teaching stylesformal/mixed/informal

adjusting for pre-testsignificant differences found

formal better than informal


Teaching Styles Study Reanalysis (Aitkin et al, 1981)

Statistical modelling approach to each stage

model based clustering of teachers — latent class model

variance component model for pupil performance and teaching style

pupil personality — normal mixture models, factor model

Involved

software development

extensive model fitting (maximum likelihood)

model selection/comparison

model validation


Clustering Teachers — Latent Class Model

Mixture model with conditional independence of items within each class

P(X = x) =K∑

j=1

πj

38∏l=1

P(Xl = xl |j , θjl)

gives probabilistic assignment of teachers to clusters

Estimation — EM algorithm

Number of classes? Selected 3 class modelfit sequence of models (multiple maxima)non-standard testing problem: bootstrap simulation for −2 log `Bayesian approach (Aitkin and Rubin, 1985)graphical tests based Total Item Score — normal mixtures

Conditional independence?

Discrete classes vs continuous latent variable (factor) models (Bockand Aitkin, 1981)


Teaching Style & Pupil Progress

Variance component model — pupils nested within classes (Teachers)

Ypqr = µ+ γxpqr + αp + Tq + εpqr

xpqr pre-test score;γ common pre-test effect → γp interaction with method

αp method (teaching style) effects

Tq teacher ability: Tq ∼ N(0, σ2T )

εpqr ∼ N(0, σ2E ) independent of Tq

Method effects found not to be statistically significant — betweenclass (teacher) variation

Model estimation used probabilistic assignment of Teachers to methods(clusters)


Teaching Style & Pupil Progress (ctd)

Could combine Teacher and method effects

Tq ∼ Mixture-N(αp, σ2T ) ≈

3∑p=1

πqpN(αp, σ2T )

where πqp are given by teacher clustering probabilistic assignments.

sort of heterogeneity model (Verbeke and Lesaffre) with fixed πqp andcommon variance; σ2

T ,p?

separation of estimation of teacher style and pupil performancejustified by conditional independence argument

joint modelling — mixture of experts heterogeneity model

models variation in pupil performance allowing for varying teacherability (variance components model)models variation in teacher ability by indicators of style (model formixing proportions)


Pupil Personality Clustering

15 personality variables (only 8 used in original analysis)Multivariate normal mixture model with common variance matrix

K∑j=1

πj N(µj ,Σ)

EM algorithm in GENSTAT; extremely slow convergence

Σ diagonal — conditional independence; multiple maxima

Normal factor model

X|U ∼ Np(µ+ ΛU,Ψ), U ∼ Nr (0, I)

Problems with normality assumptions — marginal distributionsextremely skew

Use personality variables directly in pupil achievement model —computational limitations


Teaching Styles Analyses Today

Statistical Modelling tools much improved

Computer power on a laptop

For example, in R . . .


Teaching Styles: Latent Class Models

class3x<-poLCA(f,styles,nclass=3,nrep=10) ### try a number of startsModel 1: llik = -9789.05 ... best llik = -9789.05Model 2: llik = -9790.939 ... best llik = -9789.05Model 3: llik = -9789.05 ... best llik = -9789.05Model 4: llik = -9789.05 ... best llik = -9789.05Model 5: llik = -9797.303 ... best llik = -9789.05Model 6: llik = -9789.05 ... best llik = -9789.05Model 7: llik = -9789.05 ... best llik = -9789.05Model 8: llik = -9799.223 ... best llik = -9789.05Model 9: llik = -9790.939 ... best llik = -9789.05Model 10: llik = -9798.57 ... best llik = -9789.05.......

Estimated class population shares0.3975 0.3012 0.301

Teaching Styles paper:

Estimated class population shares0.366 0.312 0.322

Variance Components Model: English

Linear mixed model fit by maximum likelihoodFormula: eng.s.p ~ eng.scen + (1 | teacher)

Data: pupilfAIC BIC logLik deviance REMLdev

6224 6243 -3108 6216 6222Random effects:Groups Name Variance Std.Dev.teacher (Intercept) 10.746 3.2781Residual 46.730 6.8360Number of obs: 920, groups: teacher, 36

Fixed effects:Estimate Std. Error t value

(Intercept) 106.45824 0.59410 179.19eng.scen 0.74348 0.01777 41.84

NPMLE Variance Components Model: EnglishFinite mass point distribution for Teacher effect Tq

No. points −2 log `

1 6325.7312 6212.7123 6206.9424 6206.6495 6206.656

Call: allvc(formula = eng.s.p ~ eng.scen, random = ~1 | teacher,data = pupilf, k = 3, random.distribution = "np")

Coefficients:eng.scen MASS1 MASS2 MASS3

0.7448 102.1820 105.2298 110.3676

Component distribution - MLE of sigma: 6.828Random effect distribution - standard deviation: 3.21772

Mixture proportions:MASS1 MASS2 MASS3

0.2230806 0.4065043 0.3704151-2 log L: 6206.9

Variance Components Model with Methods: English

Linear mixed model fit by maximum likelihoodFormula: eng.s.p ~ eng.scen + z2 + z3 + (1 | teacher)




(Intercept) 105.42793 0.86798 121.46eng.scen 0.74287 0.01773 41.89z2 -0.77508 3.48716 -0.22z3 2.19786 1.22388 1.80

Variance Components Model with Methods: Mathematics

Linear mixed model fit by maximum likelihoodFormula: math.s.p ~ math.scen + z2 + z3 + (1 | teacher)




(Intercept) 102.63308 1.21366 84.56math.scen 0.81115 0.02192 37.01z2 -3.60902 4.89355 -0.74z3 0.69271 1.71107 0.40

> anova(pupil.mathz,pupil.math)Models:pupil.math: math.s.p ~ math.scen + (1 | teacher)pupil.mathz: math.s.p ~ math.scen + z2 + z3 + (1 | teacher)

Df AIC BIC logLik Chisq Chi Df Pr(>Chisq)pupil.math 4 6433.7 6453.0 -3212.9pupil.mathz 6 6437.0 6465.9 -3212.5 0.739 2 0.691

English Scores

eng.s

eng.

s.p

8090

100110120130

80 100 130

●

●●

●

●

●

●●

●●

●●●

●●

●

●

●●●●●

●

11

●

●● ●

●

●●

●●

●●●●●

●

●●

●

12

80 100 130

●●●

● ●●

●

●●●

●

●●●●●●●

●●

13

●●

●●●

●

●

●● ●●

14

80 100 130

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

21

● ●

●●●

●

●●●

●

●

●

● ●●●

●●

●● ●

●

●

●

●●

●

●●●

●

22

80 100 130

●●●

●●●

●●●

●●

●●

●● ●●

●

●

●

●●

●● ●

●●●

●

●

●

●●

●●●

23

●

●

●●

●

●●●

●

●●

●●● ●●●

●

●

●●

●

●●

24●

● ●

●●

●●

●

●

●

●●

●

●

●

● ●●

● ●

31

●

●

●

●●●

●

●

●●

●●

●

●●

●

●●●

●

●●

●

●

32

●

●●

●

●●

●

●●

●●

●

●

●

●

●

●

●●●

●●

●●

●●●

33●

●

●●

●●

●

●

●

●

●●●

●

●●●

●●●

●●●

●

●

●

●●●●●

●

●

●●

34●

●

●

●●●

●●

●●●

● ●●

●

●

●

●

●●

●●●

●●

●

35

8090100110120130

●

●●● ●

●

●●

●

●●●●●

●●

●

●

●●

●

● ●●●

●●●●

378090

100110120130

●

●

●●

●

●●

●

●

●

●●

●

41

●

●

●

●

●●●

●

●

●

● ●●●

●

●

●●

●●

●

●●

●

●

42

●

●

●

●●

●●

●●

●

●●

●

●

●

●●

●

●●●

●

●

●

●

43●

●

●●●

●

●

●●●

●●

●●

●●●

●

●●●

●

●

●●●

●

44

●

●●

● ●

●● ●

●

●

●

●●

●

●●

●●

●●

● ●

45

●

● ●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●●●

●

●

●

●●

46

● ●●

●●●

●

●

●

●●

●

●

●●

●

●●

●

●●

●●

●●

●

●

●

51

●

●

●

●●

●

●●

●●

●

●

●●

52

●●●●

●

●

●●

●●●● ●●●

●

● ●

●●

●●

●●

●●

●

●

53

●

●

●

●

●●

●●

●●

●

●●●

●

●

●●

●●

●

54

●●●●●

●

●●

●

●

● ●

●

●

●

●

●●●

●

●

●

●

●

●

●

● ●

55●

●● ●●

●

●●

●●

●

●●

●●●●

●

●●

●

●●

●

●

●●●●●

56

●●

●●

● ●●●●●●

●●●

●

●●

●

●

●

●

●●●●

57

8090100110120130

●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●

●●●

●

●

●

●●●

●

●

●●

●

618090

100110120130

●

●●●

●● ●

●

●

●●●

●

●●

●

●

●●

●

●

●●

●

●

●●

●

●

●●

●

●

62

●

●

●●

●

●

●

● ●

●

●

●

●●

●●●

●

●

●●●

●

●●●

●●●

●●●●

●●● ●

63

●●

●

●●

●●

●

●●

●

●

●●

●● ●●●

●●●

●●

●

●

●

64

●● ●●

●●●●●

●●

●●

●●

●●

●●

●● ●

●●

●●●

●●

●

65

●●●●

●

●●●●● ●

●●

●

66●

●●

●●●●

●

●

●

●●

● ●●●

●

●●

●●●●

●●

●●

●

●

71

●●

● ●

●●

●●

●

●●

●●●●●

●●

●

●●

●

●

●

72

●

●●

●

●●

● ●●

●●●

●

●

●

●

●●

●●

●●

●●

●●● ●●

73

80 100 130

8090100110120130

●●

●

●

●

● ●●●

●

●●

●

●

●●

●

●

●●

●●

●●●

●

●

74


Mathematics Scores

math.s

mat

h.s.

p

8090

100110120130

80 100 130

●

●

●

●

●

●●

●

●●

●●●●

●●

●●

●

●

●●

●

11

●

●

●

●

●

●●●

●

●● ●

●●

●●●

●

12

80 100 130

●

●

●●●

●●

●

●

●

●

●

●●●

●

●●

●●

13

●●

● ●●

●●

●

●●●

14

80 100 130

●●

●

●●●

●

●●●

●

●

●

●

●

●

●●

●●●

●

21

●●

●

●●

●

●●

●●

●

●●

●●●

●●

●● ●

●

●●

●●

●

●●●

●

22

80 100 130

●

●●●

●

●

●

●

● ●●●

●

●

●

●

●●●

●●●

●● ●●

●●

●

●

●

●●

● ●●

23

●

●

●●

●

●●

●●● ●●●

●

●● ●

●●

●●●

●

●

24

●●●

●●● ●

●

●

●●

●

●●

●

●

●●

●

●

31

●

●

●●●●●

●

●●

●

●

●

●

●

●●

●●

●

●●

●

●

32

●

●●

●●

●

●

●●

●●

●

●

●

●

●●● ●

●

●

●●●

●

●

●

33

●●●

●

●●

●

●

●

●●●

●

●

●

●●

●●●

●●●

●

●

●

●●

●

●

●

●

●●●

34●

●

●

●

●

●

●●

●

●

●

●●

● ●●

●

● ●

●●●●

●●

●

35

8090100110120130

●●●

●●●

●

●

●●

●●

●●

● ●

●●

●

●●

●

●●●●●●

●

378090

100110120130

●

●

●

●

●

●●

●

●

●●●●

41

●●

●

●

●●●

●●●

●●●

●

●●●

●●

●●

●

●●

●

42

●●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●●

●

●

●

43

●●

●

●

●

●●

●●

●

●●

●●

●

●

●

●

●●

●●

●

●● ●

●44

●

●●●

●

● ●

●●

●●

●●

●

●

●

●●

●

●

●

●

45

●

●●

●●

●●

●

●

●

●

●●

●

●

●

●

● ●●

●

●

●

●

●

●

●●●

●

●

●

●●

46

●●●

●●●

●●

●

●

●

●

●● ●

●

●

●

●

●

●●

●●

●

●

●

●

51

●

●

●●●●

●●

● ●●

●●●

52

●●●

●●●●

●●

● ●●●

●

●

●

●

●●●

●●

●

●●●

●

●

53

●

●

●

●

●●

●●

●●

●●●

●●●

●●●

●

●

54

●●●

●

●

●

●

●●●●●●

●●●●●

●●

●

● ●

●●

●●

●

55

●●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●●●

56

●● ●

●● ●●● ●

●●

●●

●

●

●●

●

●●● ●●

●

●

57

8090100110120130

●

●●●

●●●●●

●●●

●●●

●

●●

●●●

●●●

●

●●●

●●

●●

●

618090

100110120130

●

●●

●●●

●

●●●

●●

●

●●

●●●

●●

●●●

●

●

●

●●

●●

●●●

62

●

●● ●

●

●●●

●

●●

●

●

●

●

●●●

●

●

●●

●●

●●

●●

●●

●

●

●

●● ●

●

63

●●

●

●

●●●

●●●●

●

● ●●●

●●●

●

●

●●

●●

●

●

64

●● ●●

●●

●●

●

●

●

●●

● ●●

●●●

●● ●

●●

●●

● ●●●

65

●●●●

●

●●●● ●

●

●

●●

66

●

●

●●●●

●

●

●

●●

●●

●

●●●

● ●●

●●

●●●

●● ●

●

71

●

●●●●● ●●

●

●●●

●

●

●

●●

●

●

●

●●

●

●

72

●●

●

●

●●●●

●●●

●

●

●

●

●

●●

●

●●●

●●

●

●

●

●

●

73

80 100 130

8090100110120130

●●

●

●

●

●●●

●

●

●●

●

●

●

●

●●●

●●

● ●

●●

●●

74


Reading Scores

read.s

read

.s.p

8090

100110120130

80 100 130

● ●

●

●

●

●●

●

●●

●●

●●

●●

●

●●●●

●

●

11

●

●

●

●

● ●●●

●

●●●

●

●

●

●

●

●

12

80 100 130

●●●●

●

●

●

●●●●

●

●

● ●●●●

●

●

13

●●

●

●●

●●

●●

●

●

14

80 100 130

●● ●

●

●●●

●●

●

●

●

●

●

●

●●

●

●

● ●

●

21

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●●

●●

●

●

●

●

●

●

●

● ●●●

22

80 100 130

● ●●●

●

●

●

●●

●●●●

●● ●●● ●●

●●

●

●

●●● ●

●

●

●●

●

●●●

23

●

●

●

●●●●●

●●●

●●●

●● ●●

●●●●

●●

24

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

31

●

●

●●●●●●

●●●●

●●●●●

●●

●

●●

●

●

32

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

● ●●

● ●

●

●●

●

●

33

●

●●●

●

●

●●

●

●●

●● ●

●●●

●

●

●

●● ●

●

●

●

●

●

●

● ●

●●●●

34

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●●

●

35

8090100110120130

●●●●● ●

●●●●

●●●

●

●●

● ●

●●●

●●●●●●

●●

378090

100110120130

●

●

●●

●

●

●●

●●

●

●●

41

●

●

●

●

●

●●●●

●

●●● ●

●●

●●●

●●

●

●●

●

42

●

●

●

●●●

●

●●

●

●●

●

●

●

●●

●●

●●

●

●●

●

●

43

●●

●●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●●

●●●

●

●

●

44

●

●●

●

●

● ●● ●

●

●

●●

●

●●

●

●

●

●

●●

45

●

●

●●●

●

●●

●

●●

●●

●

●

●

●●●●

●

●

●

●●

●

●●●

●

●

●

●●

46

●●●

●●●

●

●

●

●●

●

●●

●●

●

●● ●

●

●

●

●

●●

●

●

51

●

●

●

●

●

●

●●

●●

●

●

●●

52

●●●

●

●

●

●●

●●

●●●●●

●

●●

●●

●●

●●

● ●● ●

53

●

●

●●

●●

●

●

●

●●

●●

●

●●

●●●

●

●

54

●●●●●

●

●

●●

●●

●

●

●●

●●

●●

●

●

●●

●

●

●●

●

55●

●●●●

●

●● ●●

●

●●

●● ●●

●

●●

●

●●

●

●

●●●

●

●

56

●●

●

●

●

●●

●

●●

●

●

●●

●

●●

●●

●

●

● ●●

●

57

8090100110120130

●●

●●

●●●

●●●●● ●

●

●●

●●●

●●● ●

●●●●

●●●

●

●●

618090

100110120130 ●

● ●●●

●

●● ●●

●

●●

●●

●●

●●

●●

●

●

●

●●●

●●

●●

●●

62

●

●

●●

●

●

● ●●

●

●

●

●

●

●●

●

●

●●

●●

●●

● ●

●●

●●●

●●●●●

●

63

●●

●●●

●

●●

●● ●

●

●

●●

●●

●

●●

●

●●

●

●

●

●

64

●●

●●

●

●

●

●

●

●

●

●●●

●●●

●

●

●● ●

●

●

●●

●● ●●

65

●

●

●

●

●●●

●●

● ●●●

●

66●

●●

●●●

●

●

●

●●

●

●●

●

●

●

●

●●

●●●●●

●●

●●

71

●

●●●●

●

●●●

●●●

●●

●

●

●●

●

●●●

●

●

72

●●

●●●

●●●

●●●

●

●

●

●

●●

●

●

●● ●

●●

●

●●

●

●

73

80 100 130

8090100110120130

●

●

●●

●

●

●

●●

●

●●

● ●●●●

●

●●

●

●

● ●

●

●●

74


Variance Components Model: All Three Subjects

Linear mixed model fit by maximum likelihoodFormula: post ~ subject + subject:pre - 1 + (subject - 1 | teacher/pupil)

Data: pupil.allAIC BIC logLik deviance REMLdev

19395 19509 -9679 19357 19374Random effects:Groups Name Variance Std.Dev. Corrpupil:teacher subjecteng 34.580 5.8805

subjectmath 45.085 6.7145 0.341subjectread 36.064 6.0053 0.384 0.444

teacher subjecteng 11.907 3.4506subjectmath 23.867 4.8854 0.755subjectread 23.394 4.8368 0.666 0.701

Residual 13.908 3.7293Number of obs: 2849, groups: pupil:teacher, 950; teacher, 37


subjecteng 106.39793 0.61328 173.49subjectmath 102.76120 0.84360 121.81subjectread 105.12937 0.82992 126.67subjecteng:pre 0.66580 0.01712 38.88subjectmath:pre 0.73812 0.02093 35.27subjectread:pre 0.66661 0.01595 41.81

General Observations

Computing issues — time; space; software

EM algorithm — powerful general framework, but slow and nostandard errors

Random effects — normal distribution

Model based approach — preferable to ad hoc data reduction,clustering, etc

Conditional independence — a fundamental principle in modelbuilding

Model validation — limited by compute power

Constraints on modelling software capability force thinking over fitting

Murray’s modelling legacy . . .


EM Algorithm

Mixture applications of the EM algorithm in GLIM.

NPML estimation of the mixing distribution in general statistical models

Likelihood and Bayesian analysis of mixtures

A hybrid EM/Gauss-Newton algorithm for maximum likelihood in mixturedistributions

A general maximum likelihood analysis of measurement error in generalizedlinear models

Estimation and hypothesis testing in finite mixture models

Mixture models, outliers, and the EM algorithm

Marginal maximum likelihood estimation of item parameters: Application ofan EM algorithm


Random Effects

Regression models for repeated measurements

A general maximum likelihood analysis of overdispersion in generalized linearmodels

Random effect extensions of generalized linear models.

A general maximum likelihood analysis of variance components ingeneralized linear models

Regression models for binary longitudinal responses

Variance component models for longitudinal count data with baselineinformation: Epilepsy data revisited.

Random coefficient models for binary longitudinal responses with attrition.

Variance component models with binary response: Interviewer variability.


Modelling

Statistical modelling issues in school effectiveness studies

A note on the regression analysis of censored data

Statistical modelling: The likelihood approach.

Modelling variance heterogeneity in normal regression using GLIM.

Model choice in contingency table analysis using the posterior Bayes factor

Meta-analysis by random effect modelling in generalized linear models.

The fitting of exponential, Weibull and extreme value distributions tocomplex censored survival data using GLIM

Fitting the multinomial logit model with continuous covariates in GLIM

An analysis of models for the dilution and adulteration of fruit juice

Statistical modelling of unemployment rates from EEC labour force survey

A reanalysis of the Stanford heart transplant data

Bayesian model comparison and model averaging for small-area estimation

Still-births among the offspring of male radiation workers at the Sellafieldnuclear reprocessing plantJohn Hinde (NUIG) 15 April 2010 23 / 24

References

Aitkin, M, Francis, B, Hinde, J, Darnell, R (2009)Statistical Modelling in R, Oxford.

Also . . .

Aitkin, M, Francis, B, Hinde, J (2005)Statistical Modelling in Glim4, 2nd Edition, Oxford.


Documents

Teaching Styles Revisited 30 Years of Modelling · 2010. 4. 13. · Teaching Styles Revisited | 30 Years of Modelling John Hinde Science Foundation Ireland Funded Bio-SI Project &