View
37
Download
0
Category
Preview:
DESCRIPTION
Staging Abstraction using Chains of Models. …and… the Problem of Context-Dependency. Bruce Edmonds Centre for Policy Modelling Manchester Metropolitan University. Talk Outline. Introduction: a fundamental difficulty! Problem 1: needing both relevance and rigour - PowerPoint PPT Presentation
Citation preview
Staging Abstraction using Chains of Models
Bruce EdmondsCentre for Policy Modelling
Manchester Metropolitan University
…and…the Problem of Context-Dependency
Talk Outline
• Introduction: a fundamental difficulty!• Problem 1: needing both relevance and
rigour• Possible strategy: using chains of
models to stage abstraction• Problem 2: context-dependency• :Possible strategies: reducing scope and
including context• Concluding discussion: connecting the
worlds of policy and scienceStaging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-2
The Anti-Anthropocentric Assumption
• That the universe is not arranged for our benefit (as academics)
• e.g. that assumptions like the following are likely to be wrong:– Our planet is the centre of the universe– Planetary orbits are circles– Risky events follow a normal distribution– Humans act as if they followed a simple utility optimisation
algorithm• The one that I am particularly arguing against here is
that our brains happen to have evolved so as to be able to understand models adequate to the phenomena we observe
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-3
Versions of this assumption• Whilst other animals have severe limitations
and biases in their cognition, we don’t• That our tools (writing, computers etc.)
allow us to escape our limitations and biases to achieve general intelligence
• That simplicity (that which is easier for us to analyse) is any guide to truth
• If your model is not simple enough to analyse and understand, you are: (1) not clever enough, (2) lazy (have not worked hard enough), (3) premature (don’t yet have the tools to crack it) or (4) mistaken
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-4
Living with the AAA• Accepting that that much of the world around us is
fundamentally beyond capturing in a model that is both adequate and sufficiently simple and general for us to cope with
• Acknowledging our (brain+tools) biases and limitations and so considering how we might extend our scientific understanding as much as possible
• Phenomena that are simple enough for us to scientifically understand are the exception – the exception to be sought and struggled for
• Simplicity is the exception – a science of non-simple systems makes no more sense than a science of non-red things
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-5
Needing both rigour and relevanceProblem 1:
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-6
A Dilemma• KISS: Models that are simple enough to understand
and check (rigour) are difficult to directly relate to both macro data and micro evidence (lack of relevance)
• KIDS: Models that capture the critical aspects of social interaction (relevance) will be too complex and slow to understand and thoroughly check (lack of rigour)
• But we need both rigour and relevance• Mature science connects empirical fit and explanation
from micro-level (explanatory and phenomenological models)
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-7
KISS vs. KIDS as a search strategy
Simplest Possible
More Complex in Aspect 2
etc.
More Complex in Aspect 1
KISS
KIDS
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-8
The Proposed Approach
• Not to use a single model but rather a closely related “chain” of models
• Starting with narrative and statistical evidence for the micro-level behaviour of individuals etc.
• To build models that are more adequate to the processes that are thought to occur
• Which are checked and assessed against as many kinds of evidence as possible (including macro statistical evidence)
• And only later abstract to simpler simulation, analytic and social network models
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-9
Social Complexity of Immigration and Diversity
• A 5 year EPSRC-funded project between:• University of Manchester
– Institute for Social Change• Ed Fieldhouse, Nick Shryane, Nick Crossely, Yaojun Li,
Laurence Lessard-Phillips, Huw Vasey– Theoretical Physics Group
• Alan McKane, Tim Rogers• Manchester Metropolitan University
– Centre for Policy Modelling• Bruce Edmonds, Ruth Meyer, Stefano Picassa
• Aim is to apply complexity methods to social issues with policy relevance
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-10
The Modelling Approach
Data-Integration Simulation Model
Micro-Evidence Macro-Data
Abstract Simulation Model 1
Abstract Simulation Model 2
SNA Model Analytic Model
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-11
An example of the layering of related models in chemistry from 1990
• Adapted from: Gunsteren, W. F; Berendsen, H. J. C. (1990) Computational Simulation of Molecular Dynamics: Methodology, Applications and Perspectives in Chemistry. Angewandte Chemie - International Edition in English, 29:992-1023.
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-12
Roles of each kind of model
• Each is constrained by those “beneath” them, i.e. are consistent with them
• What each component should clearly represent something
• Models “above” analyse, check and explain what is happening in those below
• Models immediately “below” can be used to explore the safety of assumptions
• It might well happen that simpler, more abstract models have validity (w.r.t. a lower model) only under some settings
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-13
But why not just jump straight to simple models?• There are many possible models and you don’t know
why to choose one rather than another, this method provides the underlying reasons
• Much social behaviour is context-specific, and this approach allows one to check whether a particular simple model holds when background features/assumptions change
• The chain of reference to the evidence is explicit, allowing one to trace their effect and possibly better criticise/improve the model
• This approach facilitates the mapping onto qualitative stories/evidence
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-14
Data Integration Models
• Intended more as a computational description of a particular case than a theory (at least a general theory)
• Its aim is to represent as much of the relevant evidence as possible in one coherent and dynamic simulation
• Provides a precise target for abstraction (which are then checkable against it)
• Stages abstraction from data to theory• Separates representation and abstraction• Preserves chains of reference
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-15
Aims and Objectives of DIM
• To develop a simulation that integrates as much as possible of the relevant available evidence, both qualitative and statistical (a Data-Integration Model – a DIM)
• Regardless of how complex this makes it• A description of a specified kind of situation (not
a general theory) that represents the evidence in a single, consistent and dynamic simulation
• This simulation is then a fixed and formal target for later analysis and abstraction
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-16
DIM Development Method• A relatively tight interactive “loop” between the social scientists
who are experts in the subject matter and their data and the simulation developers...
• ...trying to give as much ownership and control to social scientists as possible.
• First target: What makes people vote (within the context of a diverse community)?
• Started with developing a fairly complete list of “causal stories” concerning the various processes that might contribute from
• Then initial model iteratively developed in NetLogo to enable maximum responsiveness and transparency
• To be reimplemented in Java/Repast when the target becomes more “settled” for more extensive simulation exploration and analysis
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-17
An overview of model structureUnderlying Data from Surveys about Population Composition etc.
Demographics of people in households (both native and immigrant)
Homophily effects the social network and membership of organisations etc.
Social network effects how individuals influence each other, reinforcing and/or changing existing norms/opinions
This effect the behaviours of individuals, which can then be extracted from the simulation as model results and compared with evidence etc.
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-18
Demonstration Run
Parametersand
Controls
Pseudo-narrative log of eventshappening to a single agent
SimpleStatistics
concerningOutcomes
Pictureof World
Indicative
Graphsand
Histograms
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-19
Example Output – Turnout
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-20
Example Output – one agent1945: (person 712) did not vote1946: (person 712) started at (workplace 31)1947: (person 712)(aged 29) moved from (patch 4 2) to (patch 5 3) due to moving to an empty home1947: (person 712) partners with (person 698) at (patch 5 3)1950: (person 712) did not vote1951: (person 712) separates from (person 698) at (patch 5 3)1951: (person 712)(aged 33) moved from (patch 5 3) to (patch 4 2) due to moving back to last household after separation1951: (person 712) did not vote1952: (person 712) partners with (person 189) at (patch 4 2)1954: (person 712)(aged 36) moved from (patch 4 2) to (patch 23 15) due to moving to an empty home1955: (person 712) did not vote1964: (person 712) started at (activity2-place 71)1964: (person 712) voted for the red party1966: (person 712) voted for the red party1970: (person 712) voted for the red party1971: (person 712) started at (workplace 9)1974: (person 712) voted for the red party1979: (person 712) voted for the red party1983: (person 712) died at (patch 23 15)
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-21
Social Network at 1950
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-22
Social Network at 1980
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-23
Social Network at 2010
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-24
On-going research and issues
• Suggests that this approach might be able to include and integrate qualitative evidence alongside quantitative evidence, but the method to do this is not well developed
• Creating and maintaining chains of models takes a LOT of time, resources etc.
• Does allow a more principled abstraction to physics type models, since at least some of the assumptions can be tested
• New trade-offs will no doubt be revealed!Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-25
Modelling and Context-DependencyProblem 2:
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-26
Another modelling trade-off
• Some desiderata for models: validity, formality, simplicity and generality
• these are difficult to obtain simultaneously (for complex systems)
• there is some sort of complicated trade-off between them (for each modelling exercise)
simplicity
generality
validity
formality
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-27
Another Dilemma!
• In order to meaningfully model, communicate or apply knowledge it has to be valid in more than one specific situation
• Yet since the AAA rules out accessible models that have general applicability…
• …we are stuck with models that seem valid and are comprehensible only in specific contexts
• One response is to make fairly simple models that give the perception that they have considerable generality, but in fact are only useful as elaborate analogies
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-28
The (direct) modelling relation
Object Systemknown unknown
Modelinput
(parameters, initial conditions etc.)
output(results)
encoding(measurement)
decoding(interpretation)
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-29
Analogies• Analogies are usually verbal, but can also be formal
(equations, simulations, etc.)• Their mapping to what is being considered is built “on
the fly” for each situation• Analogies seem to be very basic to the way humans
think and communicate• Their mapping to the situation is different for each
context and each person (in contrast to a model where the mapping is defined)
• This is done automatically and largely unconsciously• This gives the illusion of generality
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-30
Modelling a concept of something
Object System
conceptual model
Model
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-31
This part of the talk argues for the following strategy: weakening the generality of our formal models to achieve
more validity in the face of the AAA… or, to put it another way, against the following strategy: weakening validity (e.g. to analogy) to preserve
(the illusion of) generality
What is Essential to (empirical) Science?
• Validity: agreement of models to what we observe (the evidence), not science otherwise
• Formality: formal models (maths, simulation) are precise and replicable – essential to being able to build knowledge within a community of researchers
• Simplicity: ability to analyse/understand our models, good to have but unattainable in general (AAA)
• Generality: the extent of the applicability/scope of a single model, there needs to be some small generality to apply models in places other than where developed, but wide generality not necessary
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-32
Context
• “Context” is used in many different senses across different fields
• The senses and concepts herein come from discussions and papers presented at the international series of conferences on “Modelling and Using Context”
• Somewhat of a “dustbin” concept resorted to when more immediate explanations fail
• Problematic to talk about, as it is not obvious that “contexts” are identifiably distinct
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-33
Situational Context• The situation in which an event takes place• This is indefinitely extensive, it could include
anything relevant or coincident• The time and place specify it, but relevant
details might well not be retrievable • It is almost universal to abstract to what is
relevant about these to a recognised type when communicating about this
• Thus the question “What was the context?” often effectively means “What about the situation do I need to know to understand?
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-34
Linguistic Context• This is the set of all language that precede or surround
a focus utterance or phrase (the linguistic subset of the situational context)
• E.g. what pronouns might refer to• Historically the last resort of the linguist when trying to
pin down meaning• Now thought central to natural language production
and understanding• Can be extensive, relying on distant texts or linguistic
norms learnt previously• Sometimes includes common knowledge needed to
distinguish meaningStaging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-35
Cognitive Context (CC)• Many aspects of human cognition are context-
dependent, including: memory, visual perception, choice making, reasoning, emotion, and language
• The brain somehow deals with situational context effectively, abstracting kinds of situations so relevant information can be easily and preferentially accessed
• The relevant correlate of the situational context will be called the cognitive context
• It is not known how the brain does this, and probably does this in a rich and complex way that might prevent easy labeling of contexts
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-36
The Context Heuristic• Divide the world into different sorts of situation
(hereafter simply called a context)• Learn/recognise these in a rich and “fuzzy”
manner• “Crisp” knowledge is “packaged” “within” such
for reasoning, update etc.• Makes within-context reasoning, models,
update etc. more feasible• Whilst each model has limited scope, together
they might cover more ground, albeit in a more “patchy” manner
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-37
About Context-Dependency
• Context-dependency is not relativity since contexts can be reliably recognised (and/or corrected if wrongly recognised)
• But since it might be recognised in a “fuzzy” and unconscious manner the bounds of the context may not be reifiable in crisp terms
• This is a heuristic – a strategy that may help push forward the boundaries of formal empirical science
• There is some evidence that our cognition is context-dependent in many ways which means that to a considerable extent it may be unavoidable
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-38
Why might the world we study be usefully split into such “contexts”• In some domains, e.g. ecology or social
science contexts might be co-developed over time between the entities (e.g. a niche, or social context like a lecture)
• In some others it may be the only practical way to proceed, as argued above
• In yet others our cognitive, unconscious tendency to deal with the world in terms of contexts might lead us to try and divide the world along less useful lines
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-39
Context and Causality• In almost all situations (and all social situations)
there are an unlimited number of things that could be attributed as a cause
• Related to “Causal Spread” (Wheeler); “Wild Disjunction” (Fodor); and “Embeddedness” (Granovetter)
• Without a limitation as to the scope causation makes no sense
• However given a context there are many factors that can be assumed to be insignificantly relevant and/or constant
• Thus causality makes sense given a context, since it excludes most possibilities
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-40
Attending to Context• Given that attending to context is not unscientific and is
inevitable (I argue)• Then rather than pretending to generality by using
models as analogies (only)…• …I suggest attending to and incorporating context-
dependency in our investigations (as far as this is possible)…
• …and hence pushing the “envelope” of science a further in the face of complexity
• Fortunately, computers allow us to keep track of a complication of multiple contexts and avoid premature generalisation (we no longer have to weaken validity to get formality)
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-41
Kaneko (1990)
• Exhibited a system of parallel chaotic but weakly coupled processes
• Each process seems chaotic and independent• But as system size increases, variance as a
proportion of size does not disappear• Law of large numbers does not apply
Size
Variance(scaled by size)
Globally coupled
Model with random noise
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-42
An Illustration of Masked Context-Dependency
Global models are simply uninformative when the phenomena is context-dependent
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-43
Cleveland Heart Disease Data Set – the processed sub-set usedIn processed sub-set:• 281 entries• 14 numeric or numerically coded attributes• Attribute 14 is the outcome (0, 1, 2, 3, 4)• Some attributes: age, sex, resting blood
pressure (trestpbs), cholesterol (chol), fasting blood sugar (fbs), maximum heart rate (thalach), number of major vessels (0-3) colored by flourosopy (ca)
• From the Machine Learning Repository Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-44
General Correlations (1% Sig)
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-45
Fitting a Global Model (R=56%)
Num = -0.01*age + 0.17*sex + 0.20*cp + 0.00*trestbps + 0.10*restecg + -0.01*thalach + 0.23*exang + 0.18*oldpeak + 0.16*slope + 0.43*ca + 0.14*thal + -0.60 (+/- 0.83)
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-46
Looking for Clusters in HD Data Set (Start of Process)
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-47
After Solutions Locally Evolve
Speciation of SolutionsIn some areas no solution dominatesSome Solutions Spread over area of applicability
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-48
Final Set of Clustered Solutions• Final solution set
after some time.• Still complex but
some structure is revealed
• Note presence of “fbs” despite not being globally correlated and that “chol” helped define the context space
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-49
About this approach
• Difficult to quantify the extent to which one is “cherry-picking” (overfitting) models
• This only looks at one aspect of the data – one predicted variable
• A more meaningful correlate of our cognitive contexts would cluster situations as to many different aspects
• But could inform the specification of individual-based models, encoding different behaviours for each detected cluster
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-50
A useful context is one that:– includes related models with different
goals/predictions but similar scope
Clusters of Model Scopes suggest a Context
M1 M2
M1
suggests a context
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-51
Conclusions• For several reasons context-dependency is
unavoidable• Accepting and thinking about context-dependency
need not have lead to sloppy, relativistic or bad science
• In fact it is essential, since simple systems are the special case
• It may lead to pushing the boundaries of science forward and avoiding some pitfalls
• Science being imperfect or incomplete does not invalidate its project
• We may end up with a “patch work” of locally coherent models of a lot of different kinds – physics may have to be more like zoology
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-52
From this…
Physics
Chemistry
Biology
Psychology
EcologySocial Sciences
Geography
Zoology
Reduction
Infe
renc
e
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-53
…to this!
Islands of Local Consistency
Clusters of Related Models
Weaker Modelling Relations
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-54
This raises some questions…
• Does this make us all relativists?• Does this mean that scientific knowledge is
just the same as other kinds of belief?• Does this mean that we should abandon
formal models?• Does this mean that we cannot attain useful
understanding of complex systems?• Does this mean that interdisciplinary
science is hopeless?
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-55
… to which I answer “No”• The picture I paint already represents the reality of
understanding non-simple systems• It just differs from some of the rhetoric of science, and
hence the picture and beliefs many have about science• It does have some consequences for how we do
science • Rather, accepting these realities will help us do better
science by being aware of: – hidden assumptions/context– over-generalisation– reliance on single models– Models as analogies only
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-56
Connecting the Policy and Research Worlds
Concluding Discussion:
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-57
More modelling trade-offs (different set since simplicity is out)
• Formality matters to researchers – it enables the faithful passing of models to others allowing models to be collectively developed
• What policy people want is very different, they need defensible and explicable policies that have some warrant and work (mostly)
• This is a fundamental conflict
simplicity
generality
validity
formality
micro-validity
macro-validity
AggregateData
SpecificBehavioural
Models AbstractModels
WhatPolicy
AdvisersWant
Squaring the conflict
• Maybe what we should do…• Is simultaneously develop:
1. A “dirty” mainline modelusing global methods, even though pretty näive
2. But also a context-sensitive set of detailed models that are complex but less näive
• Allowing us to deliver:1. A set of “surprise free” predictions but also…2. …A “risk analysis” saying where it will probably
succeed (for targetting of the policy) as well as where and how it might go wrong
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-59
Final Conclusions
• An acceptance of complexity• The need for multiple models using multiple
techniques for any one problem• But lacking are well-founded methods for
how to use these well together!• The awareness of context-dependency and
dealing with it appropriately• Whilst developing ways of talking to
stakeholders/policy advisors that meet their needs and let them have some control
Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-60
The End
Bruce Edmondshttp://bruce.edmonds.name
Centre for Policy Modelling http://cfpm.org
The SCID Projecthttp://scid-project.org
Advertisement!
Recommended