Computational Social Science and Complex Military Operations

Computational Social Science and Complex Military OperationsAaron B. FrankDepartment of Computational Social ScienceGeorge Mason University

BackgroundComplex military operations are those whose execution demands interagency, transnational, multidisciplinary participation and perspectives

Interdependence between autonomous organizationsFundamental diversity and disparity of social systems and beliefs about them

While the challenges posed by complex military operations have been brought to forefront of the analytic community, they are nothing new

Maximization of force vs. the continuation of policy by violent meansPolitical and military objectives are not the same and may conflict

Problems are fundamentally social and interactiveCollective and strategic interaction and communicationSocial processes, memory and learning, identity

What is Computational Social Science?

CSS is a discipline in process of formation

“Big Tent” and “Small Tent” perspectivesNot directly important to DOD analytic community, but has implications for structure of a CSS “market” of servicesKnow what you are buying!

Social or Not: Land Use/Cover Modeling

• Cellular Automata (CA) vs. Agent-Based Modeling (ABM) approaches

• Consider how land changes from rural to urban• A CA model will examine how cells on a grid

change land use types based on local conditions• Each cell asks: If my neighbors are urban than I will

become urban with probability x

• An ABM will examine how parcels of land will be converted to different types based on their owning agent’s choices• Land owners make decisions based on market

conditions, social norms, law, etc.

• Both models are of a social subjects, but are both social models?

Presenter

Presentation Notes

Is the study of social insects, e.g. bees and ants social science? Is the study of solitary humans a social sciences, e.g. cognitive psychology?

The ‘Big Tent’ Approach

Three research approaches may be considered CSS

Data mining and analyticsSocial network analysisSocial simulation

Computers are essential to the conduct of the research

Not just tools for data gathering, word processing or email machines

Presenter

Presentation Notes

In each case, doing work well requires a social science orientation What to: Collect, gather or aggregate Measure How to: Inference, Represent, Code or Group What question is being asked? What does the answer look like? Ed’s story of number 3

Simulation and the Social Sciences

Many different kinds of simulation models have been used in the social sciences

Manual or seminar simulationsMathematical simulationsComputational simulations

Different treatment of behavior, data, and theoryAll move a model forward through time

What are Agent-Based Models?ABMs are composed of autonomous, interacting units

Agents have attributes and behaviorsAttributes and behaviors can be dynamic and change as a result of feedback

ABMs have environments that provide resources and mediate interactions

Spatial, network positionsSources of wealth, power or advantage in interactions

The simulation of the individual entities computes two levels of analysis

The macroscopic state of the system and its emergent propertiesThe microscopic life history of individual agents and the environment

How it Works

Presenter

Presentation Notes

Organized, coordinated violence or activity, e.g. armies Spontaneous, disorganized/self-organized violence, e.g. riots Decentralized markets Centralized corporations Cultural symbols and subcultures – Darth Vader, Hello Kitty, Comic Con, etc.

Why use an Agent-Based Model?Allows for the observation of systems and their dynamics that result from bottom-up interactions and choicesProvides an opportunity to explore relationships and behaviors not well represented by traditional mathematical models

Relaxation of ‘heroic assumptions’Enables the simultaneous incorporation of:

HeterogeneityBounded-rationalityExplicit space or interaction topology (i.e. networks)Non-equilibrium dynamics and path dependence

Journey vs. destination

Presenter

Presentation Notes

Enables the relaxation of ‘heroic assumptions’ embedded in mathematical formalisms Perfect mixing Continuous, infinite populations Homogeneity and representative agents Rationality Stationary systems Equilibrium states

ABM and the Strategic Roots of ORSA

• Early ORSA applications were highly informal, simple models that illuminated key features of military problems• Focused on the trade-offs of alternative choices and strategic dynamics• First OR study conducted in two hours in 1940

• Formalization of ORSA into a set of methods and a scientific discipline let to split and emergence of Net Assessment• Non-linearity• Adaptation and evolution• Human and organizational factors• Context, history, and culture• Comprehensive, interdisciplinary sense of strategic balance and competition

• ABM specifically, and CSS techniques generally, provide an opportunity reintegrate ORSA and Net Assessment

Presenter

Presentation Notes

Bracken – First OR study conducted on the Battle of France, looked at fighter losses and determined it was better with evacuate British Army than try and reinforce it. Important net assessment critiques of ORSA – Eliot Cohen (Jaffe Center for Security Studies), Paul Bracken (Parameters), Stephen Peter Rosen (On Not Confusing Ourselves)

The Schelling Model• In the late 1960s Thomas Schelling noted that US

cities were racially segregated but individuals were not overtly racist

• Developed a simple model of interacting, individualchoices• Agents were coins on a chessboard, each having eight

neighbors

• Agents had one simple rule:• If fewer than three of agent’s neighbors were the same type move

to a random empty space on the board

• What happened?

The Schelling Model

Presenter

Presentation Notes

Demonstration of model in NetLogo Show checkerboard as an example of a logical solution that cannot be generated

Generated Outcomes

1 2 3 4

ABM and ResultsIdentification of outcomes and relationships that emerge from microlevel specifications of agent attributes and behavior

Allows for identification of sufficient conditionsPatterns of outcomes reveal alternative futures or possibilities resulting from a set of initial conditions

Emphasis on generative or constructive knowledgeSimulation states are computed based on specified processesDifferentiates from outcomes that are logical or mathematical solutions but cannot be arrived at by interacting agents

Presenter

Presentation Notes

Counterfactuals, and scenarios – the basis of strategy Bobbitt point about comparing the future with the past Cronin, Kilcullen do the same How do we know we made the best possible choice? Was invading Iraq a “good” or “bad” decision? Better to have a complete world to examine than a single measure that goes up or down – too complex, fighting over metrics undermines analysis, collaboration, and the actual complexity of the problem

Caveat Emptor!• ABMs are incredibly flexible, but with a cost

• “Curse of dimensionality”• Overly complex

• Data overload

• Computationally expensive• Scalability concerns• Fragile outcomes• Empirical grounding of attributes and behaviors

• ABMs are most effective when used in concert with other research and analytic methods• Allows for the generalization of patterns and behaviors characterized by individual case

studies• Allows for the explorations of dynamics that may explain relations discovered by

statistical analysis

What Makes for a Good ABM Project?Organization

Theory

Data

Implementation

Validation

Experimental Design

Customer Engagement

Presenter

Presentation Notes

Organization ABMs are multidisciplinary endeavors, get the right group Social theory, computer science, quantitative analysis, domain expertise, customer knowledge Most productive members can do two roles Construct the team carefully and know its identity Policy analysis with computers? Software development that runs on social data? Ask the customer what the final product looks like Put authority in the people closest to the customer needs Theory The most important choice of any model All opportunities and limitations follow from the representation of the system Representational choices have two paths How experts think Scientific literature and theory Both paths have challenges Suitable theory should provide parameters that can be used to represent a variety of cases or conditions Suitable theory should have parameters that of interest to customers Understand the data before settling on a representation Data ABM development should start with an examination of the available data Consistent trap is the assumption that models can be populated and parameterized after construction Valuable data will exist in case studies and behavioral descriptions Quantitative data generally has three applications Parameterization of agent population attributes and environmental variables Specification of agent rules* Target result of the simulation Data analysis and collection plans should co-develop with model Don’t confuse data and theory! Implementation ABMs may be implemented in a nearly infinite number of ways Choices navigate gaps between theoretical specification, technical performance ABM are complex systems By design they do things that are unexpected Start with simple model(s) Capitalize on mathematical models Generate statistical patterns Test ‘corner’ solutions Develop in modules and save intermediate versions Identify and test within technical trade-off space Areas not specified by theory, but affect model behavior Social scientists, experts, and users must be involved and aware of implementation choices and decisions The most valuable insights will be discovered here Validation The single most difficult issue facing modeling Validation is always contextual Valid for what purpose? Valid compared to what alternatives? Validation criteria carries powerful and subtle epistemological subtexts Is the world predictable? Do we possess agency or free-will? How should our observations be coded and interpreted? Many models and questions are not subject to empirical validation Decision-making over alternatives Find an optimal allocation of resources is not the same as identifying the consequences of different choices In this context, ABM is nothing new or mystical, no different than other models Model validation by tiers and docking Experimental Design The model is two artifacts in one A representation of a theorized world A computational object with its own properties Understand the behavior of each Search across uncertainties Unknown parameters estimates Alternative model configurations Competing theories Search for model sensitivities and expressiveness Know what outcomes the model cannot achieve as a representation! Experimental agenda may reveal more about the structure of the problem than any particular model run or set of runs Customer Engagement Know your customer At the start of any project, ask how a successful project ends Keep them engaged ABM is learning by doing Prevents doing “what you know” Makes technical issues understandable Model building is expensive Slower-better-more expensive Make this clear to sponsors Only makes sense when the problem merits the resources

What is the Frontier?• Exploitation of ABM in decision-making and analytic processes

• Formalization of behavioral theories, counterfactual analysis, and scenario generation• Methods for N of 1 problem

• Empirical specification of ABM populations• Microlevel collection of data• Microlevel specification of behavioral processes• Merging models with live datastreams

• Non-utilitarian logic• For example, deontic logic

• Technical implementation• Model instrumentation• Model parallelization and non-traditional processing

• Data storage, analysis and recovery

Presenter

Presentation Notes

Mega-scale ABMs Heterogeneous agent classes that cross units of aggregation Non-traditional hardware and compute resource exploitation Agent compression, memory management Live/streaming data processing by agents High-dimensional data visualization

Organize for the Customer Needs

• ABMs are multidisciplinary endeavors, get the right group• Social theory, computer science, quantitative analysis, domain expertise, customer

knowledge• Most productive members can do two roles

• Construct the team carefully and know its identity• Policy analysis with computers?• Software development that runs on social data?

• Ask the customer what the final product looks like• Put authority in the people closest to the customer needs

Presenter

Presentation Notes

Multi-disciplinary Know your question! Organize around it! Match seniority and experience Intellectually curious and patient Appreciation for science and technology Management interest and understanding Know your identity!

Theory

• The most important choice of any model• All opportunities and limitations follow from the representation of the system• Representational choices have two paths

• How experts think• Scientific literature and theory

• Both paths have challenges

• Suitable theory should provide parameters that can be used to represent a variety of cases or conditions

• Suitable theory should have parameters that of interest to customers

• Understand the data before settling on a representation

Presenter

Presentation Notes

Theory constitutes the first and most important choice a modeler can make Tension between representations Experts deal with and trade in nuance, complexity, detail, and FACTS (or perceived facts) Modelers deal with abstraction, simplification, and assumptions Working with experts and building models from their wisdom and expertise gets rapid buy-in Trying to convince them that your abstraction is better than their experience is a sure way to become irrelevant fast If the model built from their representation doesn’t do what they expected, they will (usually) engage, open the space for new ideas and approaches Much of the social science that has been formalized is underspecified Vali Nasr – The Shia Revival – very intelligent, thoughtful and influential book – but is not formalized Does a Revival mean that Shia will “awaken” and engage more forcefully and successful in identity politics, perhaps separatist movements or even revolutionary activity? Does a Revival mean that Shia will “awaken” and become better integrated in national and regional economies and enter into positions of social and political prominence within existing political institutions? Both? Must a model generate one outcome? Both outcomes? Contingent outcomes based on local or initial conditions? General behaviors are composed of processes that are not defined Treatment of time, space, and aggregation processes are often not mentioned Krugman spatial economy model example, Lanchester equations as a process (everything happens simultaneously), meaning of a pK (better weapon, better training, better intelligence, better C2, dumber adversary, etc.) Most social theories exist in narrative, descriptive form Converting them to algorithms is non-trivial Complex behaviors may have many different layers of rules Combinations of theories may not be internally compatible Differences in time, scale, rationality Social science are not hierarchically arrange or well-ordered Feedback between levels means causation runs multiple directions Theories/disciplines provide alternative explanations for the same behavior – e.g. economic vs. sociological explanations for collective action or mate-selection

Data

• ABM development should start with an examination of the available data• Consistent trap is the assumption that models can be populated and

parameterized after construction

• Valuable data will exist in case studies and behavioral descriptions

• Quantitative data generally has three applications• Parameterization of agent population attributes and environmental variables• Specification of agent rules*• Target result of the simulation

• Data analysis and collection plans should co-develop with model

• Don’t confuse data and theory!

Presenter

Presentation Notes

Data may reside at levels of abstraction that do not match the model Data may be interpreted in many ways – unclear how it goes into a model Data resides in narratives and case studies – model builders must have aptitude and interest in “reading” and be skilled in social sciences How should model outputs, i.e. generated data be interpreted? Can it be collected in the real-world? Do not confuse laws and theories The law of gravity is a fact! – mass/distance dependent rates of acceleration The theory of gravity purports to explain why bodies are attracted to one another…. At any moment, a better theory may come along that explains the law of gravity Laws are correlations, theories address causation Keppler’s ellipsis were a theory that explained the data

Implementation• ABMs may be implemented in a nearly infinite number of ways

• Choices navigate gaps between theoretical specification, technical performance

• ABM are complex systems• By design they do things that are unexpected

• Start with simple model(s)• Capitalize on mathematical models

• Generate statistical patterns

• Test ‘corner’ solutions

• Develop in modules and save intermediate versions

• Identify and test within technical trade-off space• Areas not specified by theory, but affect model behavior

• Social scientists, experts, and users must be involved and aware of implementation choices and decisions

• The most valuable insights will be discovered here

Presenter

Presentation Notes

Modeling building is experiential learning – building the model will reveal more about the problem than any model run or output! Get the customer involved, keep them informed of what does not work!

Validation• The single most difficult issue facing modeling• Validation is always contextual

• Valid for what purpose?• Valid compared to what alternatives?

• Validation criteria carries powerful and subtle epistemological subtexts• Is the world predictable?• Do we possess agency or free-will?• How should our observations be coded and interpreted?

• Many models and questions are not subject to empirical validation• Decision-making over alternatives• Finding an optimal allocation of resources is not the same as identifying the

consequences of different choices

• In this context, ABM is nothing new or mystical, no different than other models• Model validation by tiers and docking

Presenter

Presentation Notes

This is a hard problem First opportunities for unbounded social laboratories Experiments with actual humans are limited in size and scope We can’t kill someone’s children study the dynamics of revenge and grief Many human experiments make inferences about important political and social behaviors via proxies that may or may not be suitable Was early modern European warfare more ruthless than Ancient China? Victoria Tor-bor Hui thesis, Europeans kings lacked the ruthless demeanor and interest to consolidate power as was done in Ancient China, overdependence on unreliable mercenaries Strauss and Hitch papers – demand rereading! Suboptimality (Hitch) and Nature of validity (Strauss) of the entire OR discipline! Validate by levels of analysis: Does it get the macroscopic, qualitative feature correct? Does it get the macroscopic, quantitative features correct? Does it get the microscopic qualities correct? Does it get the microscopic quantities correct? Can the model replicate the results of an alternative model when simplified? Movement off, particular parameters, homogeneous populations, etc.

Experimental Design• The model is two artifacts in one

• A representation of a theorized world• A computational object with its own properties

• Understand the behavior of each• Search across uncertainties

• Unknown parameters estimates• Alternative model configurations• Competing theories

• Search for model sensitivities and expressiveness• Know what outcomes the model cannot achieve as a representation!

• Experimental agenda may reveal more about the structure of the problem than any particular model run or set of runs

Customer Engagement• Know your customer• At the start of any project, ask how a successful project ends• Keep them engaged

• ABM is learning by doing• Prevents doing “what you know”• Makes technical issues understandable

• Model building is expensive• Slower-better-more expensive

• Make this clear to sponsors• Only makes sense when the problem merits the resources

Documents

Computational Social Science and Complex Military Operations