Modelling cooperation in infrastructure networks

Modelling cooperative agents in infrastructurenetworks

Andreas Ligtvoet, Emile J.L. Chappin and Rob M. Stikkelman

Delft University of TechnologyFaculty of Technology Policy and Management

Energy and Industry SectionP.O. Box 5015

2600 GA Delft, the Netherlands

Abstract. This paper describes the translation of concepts of coopera-tion into an agent-based model of an industrial network. It first addressesthe concept of cooperation and how this could be captured as heuristicalrules within agents. Then it describes tests using these heuristics in anabstract model of an industrial network. The discussion addresses thequestion whether the right level of abstraction is chosen and preliminaryfindings.

1 Introduction

In order to tackle some of the large societal problems related to energy generationand use – dwindling energy reserves and the emission of carbon dioxide – largeindustrial complexes have been designed and built to provide alternatives to fossilfuels or to deal with harmful emissions. There are, for example, plans to provideindustrial areas with a new synthesis gas infrastructure, to create countrywidenetworks for charging electric cars, networks for carbon capture and storage(CCS), and even larger plans to electrify the North-African desert (Desertec).

One could label these projects complex socio-technical systems [1]. Theyare technical, because they involve technical artefacts. They are social becausethese technical artefacts are governed by man-made rules, the networks oftenprovide critical societal services (e.g. light, heat and transportation), and requireorganisations to build and operate them. They are complex, because of the manyinteracting components, both social and technical, that co-evolve, self-organise,and lead to a degree of non-determinism [2]. What makes an analysis of thesesystems even more challenging is the fact that they exist in and are affected bya constantly changing environment.

While different analyses are possible of the workings of these complex socio-technical systems in interaction with their uncertain surroundings, we focus onthe question how cooperation between actors in the socio-technical system canlead to more effective infrastructures.

Cooperation, although neglected by a large group of economists, is widelypractised in the business world [3,4]. Especially in industrial networks, coopera-tion is a conditio sine qua non:

2 Andreas Ligtvoet, Emile J.L. Chappin and Rob M. Stikkelman

– industrial networks require large upfront investments that cannot be borneby single organisations;

– the exchange in industrial networks is more physical than in other companies:they become more interdependent, because the relationship is ‘hardwired’ inthe network (as opposed to an exchange of money, goods, or services in amarket-like setting).

Industrial networks require cooperation by more than two actors, which re-quires weighing of constantly shifting interests. This research aims to contributeto understanding the different ways in which actors cooperate in complex (in-dustrial) settings. The first part of the research draws on existing literature oncooperative behaviour (section 2). The other part of the research explores theoptions for planning socio-technological systems and how the behaviour of actorsinfluences the design-space. The range of options in a complex (energy) systemrequires the use of modelling individual actors’ behaviour in response to others,for which we show agent-based modelling is an appropriate approach (section3). On the basis of an abstract infrastructure network we explore different co-operation modes (section 4).

2 Cooperation as a multi-layered phenomenon

The question is how industrial networks can most effectively coordinate theiractivities. Traditionally, economic literature mainly focused on the coordinationof the various activities by means of the competitive market and the price mech-anism (the invisible hand) [3]. Competitive ‘survival of the fittest’ has been apopular biological and game theoretical model that received a lot of attentionin management strategy literature: organisations are analysed as similar to in-dividual biological organisms that survive or perish according to how well theycompete against each other for the scarce resources needed for survival. However,this model has been recognised as oversimplified [4].

Current understanding of social dilemma’s (these are situations in which oneactor’s decision depends on the decisions of other actors) is often based on theassumption that actors/agents are rational decision makers. The paradigmaticexamples of rational choice models have a number of stylised features: there aretwo symmetrical agents, who individually compute their best strategy and maydecide to act together towards a common goal. These simplifying assumptionscontrast with the circumstances that may be faced by cooperating teams inpractice, both in their size and on the dimension of motivation, competence,architecture, activity and formation, as is shown in table 1.

Game theoretical concepts are bad predictors of actual human behaviour insocial dilemma’s: in test situations, cooperation takes place more often thanwould be expected if rationality were the only determinant for decisions [6].Cooperation had (and has) an evolutionary advantage that is still embedded inour social norms [7]. In other words, our evolutionary heritage has hard-wiredus to be boundedly self-seeking at the same time that we are capable of learning

Modelling cooperative agents in infrastructure networks 3

Dimension Paradigm example In practice

Team size Small – 2 players Variable – may be largeMotivations Payoff maximising (though

what constitutes payoff mayvary)

Heterogeneous motives

Competences Symmetrical agents Heterogeneous agents, dif-ferent types with differentskills

Architecture No communication, no hier-archy

(Costly) communication,leaders

Activity Limited strategy space (of-ten binary), strategies given

Many nuanced strategies,creativity

Formation Voluntary team formation Assignment to team

Table 1. Academic approaches to teamwork and practical examples [5]

heuristics and norms, such as reciprocity, that help achieve successful collectiveaction. Individuals can achieve results that are ‘better than rational’ by buildingconditions where reputation and trust overcome the strong temptations of short-run self-interest [8,9].

We understand cooperation as a complex and layered phenomenon that en-compasses reasoning at the actor level, being influenced by phenomena at theinter-actor (network) level and societal or cultural level. The proposed researchdoes not supersede the research that has been done by behavioural psychologistsand ethologists. What it will provide is an attempt to apply heuristic approachesto cooperation in an agent-based simulation, replacing limited market reasoning.Especially with regard to the dimensions of team size, motivations, competencesand architecture, as mentioned in table 1, this may lead to more realistic insightsin the behavioural response of agents.

3 Cooperation in agents

Agent-based models (ABMs) are particularly useful to study system behaviourthat is a function of the interaction of agents and their dynamic environment,which cannot be deduced by aggregating the properties of agents [10]. In general,an agent is a model for any entity in reality that acts according to a set of rules,depending on input from the outside world. Agent-based modelling uses agentsthat act and interact according to a given set of rules to get a better insight intosystem behaviour. The emergent (system) behaviour follows from the behaviourof the agents at the lower level.

For this research, we adapted an existing set of Java instructions (based onthe RePast toolkit) and ontology (structured knowledge database) which forman ABM-toolkit that is described in [11,12]. The toolkit provides basic functionsfor describing relationships between agents that operate in an industrial setting.Herein, agents represent industrial organisations that are characterised by own-ership of technologies, exchange of goods and money, contractual relationships,


and basic economic assessment (e.g. discounted cashflow and net present valuecalculation).

Application-specific behaviour needs to be added to run the simulation ac-cording to the modellers’ requirements. Thus, we modified the agent behaviour toexamine specific cooperation-related behaviour. The following list of behaviouralassumptions were added:

– agents create a list of (strategic) options they wish to pursue: an optioncontains a set of agents they want to cooperate with and the net presentvalue of that cooperation;

– the agents have a maximum number of options they can consider, toprevent a combinatorial explosion when the number of agents is above 10;

– the agents select the agents they want to cooperate with on the basis of(predetermined) trust relationships;

– agents can have a short term (<5 years) or long term (>5 years) planninghorizon, which determines the time within which payback of investmentsshould be achieved;

– agents can be risk-averse or risk-seeking (by adjusting discount factor inthe discounted cash flow);

– agents want a minimum percentage of cost reduction before they consideran alternative options (minimum required improvement);

– agents can be initiative taking, which means that they initiate and respondto inter-agent communication.

Part of these assumptions are to provide the agents with ‘bounded rational-ity’: Herbert Simon’s idea that true rational decision making is impossible dueto the time constraints and incomplete access to information of decision makers[13]. We assume that agents do not have the time nor the resources to com-pletely analyse all possible combinations of teams in their network. They willhave to make do with heuristics to select and analyse possible partnerships. Thisis completely in line with the heuristics approach of agent based modelling.

4 A piped infrastructures test case

We applied our model with the additional cooperation settings to a test case:a cluster of n identical agents representing industries that trade a particulargood (e.g. petroleum). For reasons of simplicity and tractability, the agents areplaced with equal distances on a circle (see figure 1). The issue at hand is thetransportation of the good: the agents can either decide to choose for the flexibleoption (i.e. truck transport) or to cooperatively build a piped infrastructure totransport the good.

Flexible commitments are contracted for a yearly period only. They are agree-ments between two agents with as the main cost component variable costs asa function of distance. The permanent transportation infrastructure is built bytwo or more agents (sharing costs) and is characterised by high initial capital


costs and relatively low variable costs per distance. The more agents participatein the building of the infrastructure, the lower the capital costs per agent.

Costs minimisation will be a dominating factor in the selection of the optimalinfrastructure. Depending on the distance and the number of agents involved, aflexible solution may be cheaper than building a fixed infrastructure. By varyingthe behavioural assumptions mentioned in the section above, we investigate towhat extent cooperation takes place and what the (financial) consequences arefor the agents.

Fig. 1. A cluster of 8 trading agents in which 3 agents have cooperated to builda pipeline.

Options One of the key problems is how to decide with whom to cooperate.In the completely uniform setting that we describe, any combination of agentsis possible. However, with a large number of agents we quickly run into a com-binatorial explosion of options to consider. Therefore, the sets of colleagues thateach agent considers to cooperate with are randomly determined (mimicking aninitial ‘social network’); the full set (n agents) as well as each individual con-nection are always considered. This way all agents have access to the optimalsolution (one infrastructure in which all agents are connected) as well as theminimal solution (a connection between all pairs).

Furthermore, agents may also accept a cooperation proposal from any agent.They will assess this proposal on the basis of net present value (which is deter-mined by risk factor, required improvement and planning horizon) and may addthis new option to their list. When the list of options exceeds a predeterminednumber (in our case 25), the options with the lowest value are discarded.


Varying the behaviour of agents With the case as described above, weperformed a parameter sweep in which we varied the following variables: numberof agents, initiative taking, risk aversion and planning horizon, to see what theeffects were on the outcome of the model. The outcome of the model can bemeasured in infrastructure usage (number of pipelines built versus used) andexpenditure or income of the agents.

An analysis of the different variables’ influence shows that the initiative tak-ing factor is of crucial importance. An example of the variation of this factor for5 agents is shown in figure 2. We see that with a low initiative setting, a num-ber of pipelines are built and a near equal number of pipelines are used. Thesepipelines are mainly built in pairs, thus only slightly more efficient than drivingtrucks. With a medium initiative setting, we see that more pipes are built, butat a later stage abandoned for more efficient pipelines with more than two agentsconnected. At the end, in all cases for the medium setting, the optimal solutionof a pipeline for all five agents is chosen. The highest level of initiative displayedshows that some pipes are built that are used by more than two agents, but theyare immediately abandoned when a more optimal solution is presented.

This first variation already shows that the effect of varying initiative in agentsdoes not have a linear response in the model. Although the agents choose theoptimal solution more often with higher initiative settings, they also create un-used infrastructure in the process (in the medium initiative setting more so thanin the high). From the viewpoint of efficient use of built infrastructure, a lowinitiative setting is preferable.

0 10 20 30 40 500

2

4

6

8

10

12

Time (year)

Use

d an

d bu

ilt n

odes

0 10 20 30 40 500

2

4

6

8

10

12

Time (year)

Use

d an

d bu

ilt n

odes

0 10 20 30 40 500

2

4

6

8

10

12

Time (year)

Use

d an

d bu

ilt n

odes

Fig. 2. A cluster of 5 agents with initiative setting of 10%, 50% and 90%(dark/blue = nodes built, light/green = nodes used)

5 Discussion

For the social sciences, explaining and understanding cooperation is still one ofthe grand challenges [14]. As pointed out in [15], for researchers of industrialclusters, the space of abstract concepts has been largely explored; the challengeis to get closer to realistic models. It is important to find the appropriate balance


between detail or ‘richness’ of the model and general applicability or ‘simpleness’.We attempted to do this in the presented model.

In these first steps of our research, a crucial parameter is the ‘initiative tak-ing’ factor or the willingness to cooperate. Especially in large groups of agentsjust one agent unwilling to cooperate can halt the process. Clearly, the chancesof all agents reaching an agreement is small when the initiative taking factor issmall. The lower the initiative factor, the less optimal solutions are pursued bythe cluster of agents and the more money is spent on infrastructure building.One could see here the difficulty of reaching consensus with a large group andthe subsequent build-up of infrastructures in small steps. We also see that be-fore reaching the optimal solution from the agents’ point of view, some form ofinefficiency cannot be avoided.

When agents attempt to analyse all possible cooperation strategies with morethan approximately 10 agents, one quickly runs into the problem of combinato-rial explosions. As with ‘real’ actors, the agents require a heuristic to limit thenumber of options they consider. Each agent starts with his own (randomised)set of strategies with preferred cooperation candidates. The agents have a maxi-mum number of strategies they consider each round. When they cooperate, theymay exchange new suggestions for strategies, of which only the most profitableare kept. In this way computational time is limited.

We thus addressed a number of the issues in table 1: although agents arepay-off maximising, their decisions are influenced by several factors, and there-fore more heterogeneous. Their strategies are combinatorial and not binary, al-though not very ‘creative’ (e.g. they do not start a totally different type of busi-ness). Different agents have different strategies that they need to communicateto achieve cooperation. And finally, the model allows for any number of agentsinvolved although adding agents will increase computing time exponentially. Forthe sake of tractability we have kept the agents symmetrical, although introduc-ing asymmetrical agents (both from the viewpoint of location and behaviouralcharacteristics) is possible in the model.

6 Conclusions

The first attempts to embed cooperative behaviour with bounded rationality inour agents has yielded interesting and unexpected results. Further investigationof factors that play a role in cooperation decisions seems appropriate.

The question, however, remains where the boundary lies between the needfor more detail and complexity of the model, and the general usability of it.Although they are elegant in their simplicity, 2x2 pay-off matrices do not ad-equately capture the issue of cooperation in a real-world setting. Conversely,introducing a plethora of factors and heuristics to enrich agent behaviour willlead to intractable outcomes and (too) long computation time. A limited set offactors (e.g. <10) is probably adequate. This paper gives a first suggestion as towhat factors have to be considered when modelling cooperative behaviour.

References

[1] Herder, P.M., Stikkelman, R.M., Dijkema, G.P., Correlje, A.F.: Design of asyngas infrastructure. In Braunschweig, B., Joulia, X., eds.: 18th EuropeanSymposium on Computer Aided Process Engineering, ESCAPE 18, Elsevier(2008)

[2] Pavard, B., Dugdale, J.: Unifying Themes in Complex Systems. In: TheContribution Of Complexity Theory To The Study Of Socio-Technical Co-operative Systems. Volume IIIB of New Research. Springer (2006) 39–48

[3] dei Ottati, G.: Cooperation and competition in the industrial district as anorganization model. European Planning Studies 2(4) (1994) 463–481

[4] Nielsen, R.P.: Cooperative strategy. Strategic Management Journal 9 (1988)475–492

[5] Gold, N.: Teamwork - a multidisciplinary perspective. In: Introduction:Teamwork in theory and in practice. Palgrave Macmillan, New York (2005)1–20

[6] Colman, A.M.: Cooperation, psychological game theory, and limitations ofrationality in social interaction. Behavioural and Brain Sciences 26 (2003)139–198

[7] Barkow, J.H., Cosmides, L., Tooby, J., eds.: The adapted mind - evolu-tionary psychology and the generation of culture. Oxford University Press,New York (1992)

[8] Ostrom, E.: A behavioral approach to the rational choice theory of collectiveaction. American Political Science Review 92(1) (March 1998) 1–22

[9] Cosmides, L., Tooby, J.: Better than rational: Evolutionary psychologyand the invisible hand. The American Economic Review 84(2) (May 1994)327–332

[10] Beck, J., Kempener, R., Cohen, B., Petrie, J.: A complex systems approachto planning, optimization and decision making for energy networks. EnergyPolicy 36 (August 2008) 2803–2813

[11] Nikolic, I.: Co-Evolutionary Process For Modelling Large Scale Socio-Technical Systems Evolution. PhD thesis, Delft University of Technology,Delft, The Netherlands (2009)

[12] van Dam, K.H.: Capturing socio-technical systems with agent-based mod-elling. PhD thesis, Delft University of Technology, Delft, The Netherlands(2009)

[13] Simon, F.B.: Einfuhrung in die systemische Organisationstheorie. Carl-AuerVerlag, Heidelberg (2007)

[14] Colman, A.M.: The puzzle of cooperation. Nature 440 (2006) 745–746[15] Fioretti, G.: Agent based models of industrial clusters and districts (March

2005)

Documents

Modelling cooperation in infrastructure networks