Interactions between the individual and the group:

reflections from multilevel modelling in educational research

Federica RussoCenter Leo Apostel, Vrije Universiteit Brussel &

Centre for Reasoning, University of Kent

2

Overview

Philosophy of education and empirical researchReverse the question: does empirical research look into philosophy?

Multilevel models in educational researchDefinition and examplesThe need for an accompanying ‘substantive theory’

A ‘substantive theory’ for multilevelRecent work by Little and YilokoskiMain features

3

PHILOSOPHY OF EDUCATION AND EMPIRICAL RESEARCH

4

Does PhilEd pay enough attentionto empirical research?

Phillips J Phil Ed 39(4), (2005)

• PhilEd hasn’t paid enough and serious attention to empirical research

• It is possible to study normative processes empirically

• Mutual benefit of PhilEd and empirical research to look into real cases

Hyslop-Margison & NaseemPhl Ed Archive (2007)

• Straw man: wrong selection of critics

• Counterexamples: PhilEd does pay attention to empirical research

• Problem of empirical generalisability

5

PhilSci and PhilEduPhillips (2005, p.582):

The marked change in doing philosophy of science came about when it was realised that there was much to gain by taking scientific research seriously, rather than discussing an artefact of the philosophers’ imagination. […]

The present essay is making a call for a parallel revolution in philosophical discussions of educational research, a revolution that entails taking examples of educational research seriously. […]Processes that humans engage in, in the real world, whether normative or cultural or psychological (or all three at once) can be studied—and probably ought to be studied—empirically, but they also need to be assessed in terms of the values (and if relevant the conception of education) that they embody.

Turn the question on its head

Does empirical research look sufficiently into philosophy of education? (Or, for the matter, into philosophy?)

7

EMPIRICAL RESEARCH

8

Modelling in the social sciences

Causal relations in social contextsMarital problems ⇄ migrationMaternal education → child survivalStress + physical health + …→ self-rated health

Two approachesQualitative: smaller and focused samplesQuantitative: statistical analyses of large data

9

MULTILEVEL MODELSA crash course

10

Why multilevel?

An example of quantitative methods used in empirical research in educationGoing quantitative, the new panacea for evidence

But … is it really panacea?

It models hierarchical structuresTypical of social (and education) contexts

A sounding board for the question: does empirical research look into philosophy?

11

Multilevel models

A special type of statistical model used in causal analysis to model hierarchical structures:

Individuals / family / local population / national populationFirms / regional market / national market / global marketPupils / classes / school / school systems

No a priori reason to choose the level of analysisActually, good reasons to study the interactions between

the levels

12

Traditional approaches

Holismthe system as a whole determines the behaviour of the parts in a fundamental way; the properties of a given system cannot be reduced to the mere sum of its components

Individualismsocial phenomena and behaviours have to be explained by appealing to individual decisions and actions, without invoking any factor transcending them

13

The ‘statistical’ counterparts

Aggregate-level models explain aggregate-level outcomes through aggregate-level

variables

Individual-level models explain individual-level outcomes by individual-level

explanatory variables

14

Types of variables

Individual: measure individual characteristics, take values of each of the lower units in the sample. e.g. income of each individual in the sample

Aggregate: summary of the characteristics of individuals composing the group e.g.: mean income of state residents

15

Dangers

Atomistic fallacywrongly infer a relation between units at a higher level of analysis from units at a lower level of analysis

Ecological fallacydraw inferences about relations between individual level variables based on the group level data

16

Robinson: illiteracy and immigration

1930 census in the US, for each of 48 states + district of Columbia

Individual correlation: descriptive properties of individualsPositive correlation: immigrants more illiterate than native citizens

Ecological correlation: descriptive properties of groupsNegative correlation: correlation between being foreign-born and

illiterate magnified and in the reversed direction

Explanation: immigrants tend to settle down in states where native population is more literate

17

Courgeau: Farmers’ migration in Norway

Data from the Norwegian population registry (since 1964) and from two national censuses (1970 and 1980)

Aggregate model and individual model show opposite results:Aggregate: regions with more farmers are those with higher rates of

migrations;Individual: in a same region migration rates are lower for farmers than

for non-farmers

Reconciliation: multilevel modelaggregate characteristics (e.g. the percentage of farmers)explain individual behaviour (e.g. migrants’ behaviour)

18

Types of models - summaryIndividual: explain individual-level outcomes by individual-level

explanatory variablese.g.: explain the individual probability of migrating through the individual

characteristics of being/not being farmer

Aggregate: explain aggregate-level outcomes through explanatory aggregate-level variables

e.g.: explain the percentage of migrants in a region through the percentage of people in the population having a certain occupational status (e.g. being a farmer)

Multilevel: make claims across the levels, from the aggregate-level to the individual-level and vice-versa

e.g.: explain the individual probability to migrate for non-farmers through the percentage of farmers in the same region

19

Grouping in multilevel

Units grouped at different levels, a-contextual language

Grouping may be more or less randomOnce the grouping is done, differentiation:

group and its member influence and are influenced by the group membership

20

Statistical modelling of hierarchies

0 1 2ij j j ij j ijY x z

response variable at the individual level

explanatory variable at the individual level

explanatory variable at the group level

i: index for the individuals

j: index for the group

these vary depending on the group

Errors are independent at each level and between levels

21

Goldstein:Multilevel in educational research

Study school effectiveness, examination results, …All quantifiable aspects of education

‘Statistical’ advantages of multilevelEfficient estimates of regression coefficientCorrect standard errors, confidence intervals,

significance tests for the clustersEnables measuring differences between clusters

http://www.math.helsinki.fi/msm/banocoss/Goldstein_course.pdf

22

Hierarchies in educational research

Simple hierarchy:Pupil / class / school / neighbourhood /

Cross-classified structurePupil – ethnicity // school – neighbourhood //

23

Goldstein et al: examination results and school differences

Inner London schools Response variable: examination resultsExplanatory variables: standardised London reading tests, verbal

reasoning category, gender, school gender (mixed, boys, girls), school religious denomination (State, Church of England, Roman Catholic, other)

Results:Small effect of school gender; Roman Catholic slightly better; girls better

than boys; large differences for different verbal reasoning categories.

Differences between schools in examination results depend on intake achievement and curriculum subject considered

No single dimension in which schools differ

24

Driessen: School composition and primary school achievement

Dutch primary schoolsResponse variable: language and math proficiencyExplanatory variables: parental ethnicity and education,

pupils sex and age, school composition, ethnic diversity

Results:Quite strong effect of school composition on language,

weak on math; all children, independently of background, perform worse in schools with high ethnic diversity

Question about distribution policy and other measures

25

[…] despite their usefulness, models for multilevel analysis cannot be a universal panacea. […]They are not substitutes for well grounded substantive theories […]Multilevel models are tools to be used with care and understanding.

Goldstein, Multilevel statistical models,http://www.bristol.ac.uk/cmm/team/hg/multbook1995.pdf

26

WHAT ‘SUBSTANTIVE THEORY’ FOR MULTILEVEL?

27

Modelling and explaining

What does a multilevel model model?Relations between different levels in a hierarchical structure

What does a multilevel model explain?How group behaviour influences individual behaviour (but not vice-versa)

Statistically, multilevel achieves bothBut the ‘substantive theory’ is still wanting

28

What needs the ‘substantive theory’

School religious denominationWhat social practices, norms, values are involved?

School composition and ethnicityHow do these influence peer relation among pupils?

…

What is the extra information that we need and that statistics does not give us?

29

LEVELS IN A SUBSTANTIVE THEORY

30

Levels, beyond statistics

Dan LittleLevels of the social:

Ontology what social entities?Explanation reduction?Causation causal powers? Inquiry what level? Description what level requirements? Generalisation recurrence of types?

Avoid analogies with natural sciences, don’t reify social phenomena

31

Levels, beyond the received views

Methodological individualism

• Social facts must be reducible to facts about individuals

• There is no higher level without lower level

• E.g.: Austrian school economics, some political scientists

Holism

• Social entities and structures have primacy and are independent

• Individuals are influenced by social facts, but do not influence them

• E.g.: sociologists in the Durkheim tradition

32

Methodological localismSocial structures influence social outcomes, embodied in action of socially

constructed individuals

Individuals are the bearers of social structures and causes, but individual actors are socially constructed

Emphasis:Contingency of social processesMutability of social structures over space and timeVariability of human social systems (norms, social practices, urban arrangements, …)

Cast doubt on generalisable theories across many populations, look for specific causal variation

33

Scale-based levels

Petri Ylikoski:Macro social facts are typically supra-individual

Micro and macro have a part-whole relationship, but not just mereological constitution

Difference in scale, not categorical

A heuristic, as there is no unique micro-level, context-relativeness

34

Different questions

Constitutive questions

• How macro properties are constituted by smaller-scale entities

• How the macro depends on the micro

• How the macro would have been different, had the micro been different

Causal questions

• Origin, persistence, and change of macro properties

• What the outcome would have been, had things in the causal history been different

35

THE SUBSTANTIVE THEORY, MAIN FEATURES

36

Levels and types of question

Give reality to levelsallocation is not random, not a statistical artefact

Articulate the embodied aspects of level interactionsSociology, anthropology, pedagogy, psychology, …

Look for empirical origins of variations in outcomeLarge-scale statistical studiesSmall-scale qualitative studiesINTEGRATION of explanation of socially-constituted behaviours

37

TO SUM UP AND CONCLUDE

38

In this talk:PhilEd and empirical research

Disputed question of the relation between PhiEd and empirical researchTurn the question on its head:

does empirical research look into philosophy?

Empirical researchCausal modelling widely used in social research, including educationSophisticated formalisms are designed to measure, model, explain social

reality, including hierarchical structuresDespite progress and improvements, formal methods are still in need of

‘substantive’ theory

In search of a substantive theoryRecent work by Little and Yilikoski addresses level ontology, it helps find the

main features of the substantive theory

39

Trouble shared, trouble halved?

Empirical research does not look outside statistics sufficientlyA problem shared also by e.g. social epidemiology

Statistical modelling, an alleged gold standard to generate evidenceA problem shared by e.g. evidence-based medicine

40

Remedies?

Qualitative research, philosophical investigations to feed empirical research

Dismantle evidence hierarchies and gold standards

Build integrated methodsQuantitative and qualitativeEmpirical and conceptualMultiple sources of evidence

41

REFERENCES

42

Courgeau D. 1994 Du groupe à l’individue: l’exemple des comportements migratoires. Population 1.

Courgeau D. 2007 Multilevel synthesis. From the group to the individual. Springer.

Driessen G. 2002 School composition and achievement in primary education: a large-scale multilevel approach. Studies in Educational Evaluation 28.

Goldstein H. 1999 Multilevel statistical models. Wiley.

Goldstein et al. 1993 A multilevel analysis of school examination results. Oxford Review of Education 19(4).

Hyslop-Margison EJ and Ayaz Naseem M. 2007 Philosophy of education and the contested nature of empirical research: a rejoinder to D.C.Phillips. Philosophy of Education.

Little D. 2006 Levels of the social. In The Philosophy of Anthropology and Sociology, Risjord and Turner (eds). Elsevier Science.

Phillips D.C. 2005 The Contested Nature of Empirical Educational Research (and Why Philosophy of Education Offers Little Help). Journal of Philosophy of Education 39(4).

Robinson W.S. 1950 Ecological Correlations and the Behavior of Individuals. American Sociological Review, 15(3)

Russo F. Causality and causal modelling in the social sciences. Measuring variations. Springer. 2009

Ylikoski P. 2012 Micro, macro, and mechanisms. In Oxford Handbook of Philosophy of Social Sciences, Kinkaid (ed) OUP