A framework for designing policies for networked systems with uncertainty

Decision Support Systems 49 (2010) 121–131

Contents lists available at ScienceDirect

Decision Support Systems

j ourna l homepage: www.e lsev ie r.com/ locate /dss

A framework for designing policies for networked systems with uncertainty

Surya Pathak a,⁎, Mark McDonald b, Sankaran Mahadevan b

a Business Program, University of Washington, Bothell, WA, 98011, United Statesb Department of Civil and Environmental Engineering, Vanderbilt University, Nashville, TN, 37235, United States

⁎ Corresponding author. Business Program, UniverBox 358533, 18115 Campus Way NE, Bothell, WA, 9801

E-mail address: [email protected] (S. Path

0167-9236/$ – see front matter © 2010 Elsevier B.V. Aldoi:10.1016/j.dss.2010.01.006

a b s t r a c t
a r t i c l e i n f o
Article history:Received 26 November 2007Received in revised form 15 January 2010Accepted 20 January 2010Available online 28 January 2010

Keywords:Policy designTransportation networkSystem of systemsOptimizationUncertaintyAgent-based modelingNetwork systemsLyapunov exponent

This paper presents a framework to design policies for networked systems. The framework integrates modelbuilding, stability analysis of dynamic systems, surrogate model generation and optimization underuncertainty. We illustrate the framework using a transportation network benchmark problem. We considerbounded rational users and model the network using software agents. We use Largest Lyapunov exponentsto characterize stability and use Gaussian process model as an inexpensive surrogate, facilitatingcomputational efficiency in policy optimization under uncertainty. We demonstrate scalability by solvinga traffic grid policy design problem and show how the framework lends itself towards carrying out stabilityversus performance tradeoffs.

sity of Washington, Bothell,1–8246, United States.ak).

l rights reserved.

© 2010 Elsevier B.V. All rights reserved.

1. Introduction

During the past decade there has been a growing interest indeveloping conceptual, methodological and analytical approaches forstudying large scale inter-disciplinary problems that are embedded innetworks at multiple levels and multiple domains [36]. Such systemstypically consist of networks that have multiple decision makers andexhibit operational and managerial independence, geographical distri-bution, and emergent behavior [15]. Modern transportation and supplynetworks, critical infrastructure networks, energy and power networks[36] display such properties. For example, transportation networks inUS metropolitan cities typically have multiple modes of transportation(roadways with passenger cars, buses, and subway and ferry system).Collections of these individual networks are used by numerous users,who display different kinds of user behavior (risk averse, risk taking,rational, bounded rational). The individual networks are managed byindependently governed units with managerial independence and areaffectedby changes in the surroundinguncertain environment resultingin distinct flow patterns (on each of these systems). Similar propertiesare observed in supply networks in which multiple independentorganizations displaying heterogeneous behavior while operatingunder an uncertain environment to fulfill customer demand, display

distinct patterns of product and information flow and changingtopological structures.

While research in the field of policy design for real life networkedsystem has evolved from the initial days of drawing influencediagrams [35], it is still considered a challenge to integrate the abovementioned dynamisms within a policy design framework that canaccommodate complex network topologies, diverse user behavior andrich set of system-user interactions [36]. In this paper we suggest anapproach that follows the same trend of model building, model analysisand decision making as used by multitude of researchers[3,7,8,24,30,34–36] and presents comprehensive additions in the formof stability analysis and surrogate model representation to betteraddress the non-linearity and dynamism in real life networked systems.The four step approach can be summarized into a logical framework as(See Fig. 1):

1. Modeling of system behavior along with user behavior: while ourframework is not limited to any particular modeling methodologywe demonstrate our example problems using agent based simula-tions to capture how system response changes as individual usersinteract with the network in order to achieve their goal, and howthe individual user behavior is impacted by the system state.

2. Analyzing system response behavior for stability conditions: recentliterature on large scale system design [3,37] suggests that optimalsolutions in networked systems should be carried out in the stableregion of operations in order for the optimal policy to be effective.Design of policy in chaotic regime defeats the purpose of anoptimal policy design approach. Thus we ensure that the set of

mailto:[email protected]

http://dx.doi.org/10.1016/j.dss.2010.01.006

http://www.sciencedirect.com/science/journal/01679236

Fig. 1. Framework for policy optimization in network systems.

122 S. Pathak et al. / Decision Support Systems 49 (2010) 121–131

input and decision variables being considered yields stable systemresponse behavior before optimizing policy variables.

3. Constructing an inexpensive surrogate for the expensive agent-based simulation: The computational effort associated with the useof agent-basedmodeling (ABM) for optimization in the presence ofuncertainty is enormous. Three types of iterative analyses arerequired: system and user simulation, uncertainty analysis andoptimization. Construction of a surrogate model reduces thecomputational expense and facilitates the estimation of expectedvalue of the system response behavior which can then be usedfurther in the optimization process.

4. Optimizing policy variables through stochastic optimization: asdescribed above network systems typically have multiple types ofuncertainties associated with them. In order to design an optimalpolicy for such systems a stochastic optimization approach wherethe problem formulation includes system response variables, theinput variables and the decision (policy) variables and theirstochastic characteristics is necessary. Use of a surrogate modelrepresentation further reduces the complexity of implementingoptimization routines as full grid searches become feasible. Wedemonstrate this with our two examples later in the paper.

In order to illustrate our framework we use a popular benchmarkproblem as a verification problem in the transportation networkliterature called the Braess network [6] that has been widely used byresearchers [12,13,25,36,37,42,53]. The Braess network consists of asingle origin-destination (OD) pair and four nodes.We later use a singlerandom input (random demand) and a single policy control variable(flow split at the origin node) to illustrate each of the four stepsmentioned above. First we use an agent-based modeling (ABM)approach for modeling the Braess network with multiple users withbounded rational behavior [54]. Second, we compute the LargestLyapunov exponent (LPE) [20,46,47,50] to analyze the stability of thesystem response behavior. Next, we estimate a closed functional formofthe overall system cost (system response behavior) using a Gaussianprocess (GP) surrogate estimation technique [43]. Finally we performstochastic optimization to find the value for the flow-split policyvariable that minimizes the total system cost in the network. We thenillustrate scalability of our framework by designing policies for a trafficgrid with 25 signalized intersections, 80 links and 10 times the numberof users as compared to the verification problem. We also introduce

stability response surfaces as a trade off toolwhile selectingpolicies for anetwork system.

While the individual tools used in this framework are proven andtested individually; to the best of our knowledge, they have never beenused together to methodically approach policy design problems fornetworked systems. The primary contribution of this paper is thesynthesis of model-based network system representation, stabilityanalysis of dynamic systems, surrogate model reconstruction andoptimization under uncertainty (OUU) within an overarching frame-work, enabling design of stable policies for network systems. Strengthsof this framework include the ability to use any modeling approach forrepresenting the system. We impose no constraints except for therequirement that a systemmodel can be used to generate training datafor stability analysis and surrogatemodel reconstruction (in the form oftime series representation of dependent and independent variables). Anovel aspect of this research is the stability analysis of the dynamicsystem responses and surrogate reconstruction of a stability surface. Asshown in the later sections with one of our complex network (ahypothetical urban traffic grid) demonstrations, stability surfaces can beused for making tradeoffs between different policy aspects in anetworked system along with ensuring that the designed policies areselected from a stable operating zone. The integration of GaussianProcess Models (GP) in this framework enables construction of thestability surface while also allowing us to use a cheap surrogaterepresentation of the system in the optimization process. Theinexpensive computation time provided by the GP model allows us touse very simple optimization routines such as line searches and full gridsearches.

Section 2 presents the background literature on network systemmodeling and simulation. Section 3 introduces the Braess network(verification problem) and policy design formulation. Section 4presents detailed step by step illustration of the agent-based modelfor the Braess system. Section 5 presents the LPE analysis whileSection 6 discusses the surrogate model construction. Section 7presents a stochastic optimization approach for finding the optimaldesign value for the policy variable. Section 8 then illustratesscalability of our framework. Section 9 concludes and suggests futureresearch.

2. Background

Researchers from diverse domains such as transportation systemdesign [13,53], supply network design [11,36], andmore recently supernetwork (overlapping transportation and supply networks) [37,42] andemergency response network design [3] have attempted to designoptimal economic and operational control policies for network systems.For example, in transportation network and supply network design, theapproaches include: system and user equilibrium behavior models(logit and probit models) [13,53], variational inequality formulationsthat consider multiple user types and multiple classes of networks(super networks), and system dynamics formulations for timedependent problems [37]. Several solution approaches such asstochastic programming, discrete time and discrete event simulationsand multidisciplinary optimization methods [24,27,40] have beenconsidered. Additionally researchers in facility network layout design[14,17,44] have considered uncertainty associated with demandgeneration, fixed costs, transportation between nodes etc. whilesuggesting numerous solution approaches for designing optimalsolutions/policies such as dynamic programming [28], lagrangianheuristics [1], genetic algorithms [4], and tabu-search [2] to name a few.

Carley and Kamneva [8] have developed a dynamic networkanalysis and optimization approach for improving organizationalpolicy design while addressing some of the above mentioned issues.They represent network systems in the form of a relationshipmatrix between the nodes of the network and use Monte Carloand simulated annealing-based heuristic optimization methods to

123S. Pathak et al. / Decision Support Systems 49 (2010) 121–131

minimize vulnerabilities (defined by underlying metrics such asworkload distribution, resource distribution etc.) in the relationshipmatrix. This approach takes a mission-driven perspective and as themissions change a new relationship matrix is fed into the optimizerwhich yields a new set of results. Mohapatra and Sharma [35] haveused an alternative approach for policy design using modal controltheory. They treat policy variables as control variables by delinkingthem from other system variables. Provided that this reduced systemis linear and controllable, it is possible to synthetically generatecontrol policies by modal control theory to ensure any prescribeddegree of stability. These theoretical control policies then can be usedto design realistic policy decisions. One of the possible limitations ofthis approach is the assumption that policy variables can becompletely delinked.

While research in these areas has advanced the state of the art inoptimal policy design for networked systems, there are limitations inthe formof fixed and static network topologies, assumption of fixed andperfectly rational user behavior (or firm behavior in the case oforganizational networks), sealed-off nature of the solutions (limitedability to combine micro-level user behavior with macro levelparameters) [21], lack of stability considerations while designing thesolutions, and mathematical intractability for larger scale problems. Inreality, network systems such as mentioned above display dynamicemergentbehavior [7] in the formof evolving topologies [26,40], diversesystem response (such as equilibrium flow patterns in a traffic networkor population dynamics in a supply network), bounded rational userbehavior [9,54] and instability and chaos [10].

Additionally, network systems have multiple sources of uncer-tainty associated with them. There are aleatory uncertainties in theform of random demand (varying number of users using atransportation network), and randomness in speed, flow or density(the three are interrelated). There is epistemic uncertainty associatedwith the information flow in the network and individual userbehavior. For example in a transportation network, informationprovided to users in the form of advanced traveler informationsystems (ATIS) may be uncertain. Also individual users lack theknowledge of how the rest of the users are behaving. Thus a userwhile taking a decision makes adjustments based on his/herperception of the state of the system and how the other users arebehaving. Thus network policy optimization needs to properlyaccount for the various sources of uncertainty in addition toaddressing the challenges mentioned above.

2.1. Model based representation

A networked system can be modeled using many differentapproaches.We discuss the pros and cons of some popular approachesin this section.

2.1.1. Equation-based approachesTraditional paradigms for modeling networks include user equi-

librium, based on the concepts of equilibrium in n-person games [38],and systemoptimality [55], inwhich the total systemcost, representedin terms of input and decision variables is minimized. Nashequilibrium assumes self-interested behavior and perfectly rationalusers, while system optimality implies that all decisions are made by acentral authority. Although both user equilibrium and systemoptimality assumptions are useful in the design and analysis ofnetwork systems, both assumptions are too restrictive. Typically innetwork-based systems the behavior is driven by multiple classes ofusers [37] who are locally optimal in their decision making and areseldom fully rational in their behavior. Thus, if the user behaviorassumptions are incorrect, the resulting behaviors observed fromchanges in network policies could be significantly different fromwhatis expected.

Another network system modeling technique is to use systemdynamics [22] that uses concepts of stocks and flows along withfeedback and time delay relationships to model the dynamic behaviorof network systems. System dynamics is extremely useful foridentifying the important variables and causal linkages in a system.Whether game-theoretic or system dynamics-based, one of the majorlimitations of the equation-based methods is their need for mathe-matical representation of both system and user behavior. The numberof decision sub-problems that need to be analyzed grows very largeand quickly becomes mathematically intractable as the underlyingnetwork system grows larger andmore complex. Alsomultiple classesof users with heterogeneous decision making capabilities furtherincrease the difficulty in the equation-based approach.

2.1.2. Agent-based approachesAgent-based modeling (ABM) is yet another technique that can be

used for representing network-based systems. An agent in an ABMmodel is a persistent entity with an internal state which interacts withother agents, mutually modifying each others' states [21,52]. Thus anagent-based model consists of a collection of agents and their states,the rules governing the interactions of the agents, and the environ-ment within which they operate. The state of an agent can bearbitrarily simple, such as position, or the color of a cell in a cellularautomaton [21,52]. States can also be extremely complicated,including, possibly, sophisticated internal models of the agent'sworld [52]. Agent-based models typically utilize a bottoms-upapproach and represent entities, their behaviors and their interactionswhile investigating emergent behavior, as compared to equation-based models that typically model aggregate system behavior. ThusABMs are verywell suited for capturing heterogeneous classes of usersand multiple autonomous decision makers displaying boundedrational behavior, typical constituents of a network-based system.ABMs have been used inmultiple disciplines such as e-commerce [29],powermarket design [57], social sciences [5], operationsmanagement[40], complexity science [52], and computer science [45], for solvingproblems with similar characteristics as described above. We utilizeagent basedmodels in this paper for representing the traffic networks.Using agents allows us tomodel bounded rational user behaviorwhichis a key attribute of a real life network system.

2.2. Simulation as a common formalism

Whether network systems are modeled using equation-basedapproaches or using ABMs they share a common problem character-istic; they have interactions among users and environment whichresults in emergent patterns over time. For example, in supplynetworks, the topology of the network emerges over time driven byfirm-to-firm interactions [40], and in transportation networks trafficflow behavior emerges over time driven by individual decisions madeby users. Simulation methods [61] are capable of capturing the timedependent structural and behavioral emergence as well as theinformation exchange between the multiple components of suchnetwork systems. Particularly, discrete-time simulation formalismcaptures the notion of time-dependent emergence of networksystems whereas discrete event formalism captures exchange ofinformation between multiple components [61]. In the case ofequation-based approaches, the discrete-time or discrete-eventformalisms are typically represented with explicit algebraic, differ-ential and/or difference equations. In the case of ABMs high levelcomputer languages provide support for modeling discrete-time (e.g.thread based design [31]) and discrete-event formalisms (e.g.,message passing [19]). In most real-world network system problemsa hybrid approach combining both discrete-time and discrete-eventformalisms is needed. One of the highlights of our framework assuggested earlier is that it is agnostic to the modeling methodology.

Fig. 2. The Braess network.


3. Example network system: Braess network

As mentioned previously we use the well-known Braess networkin Fig. 2 [6,36] to illustrate the suggested policy design meta-framework in Fig. 1. The Braess network is a well benchmarkedsystem in the transportation literature and one of the primary reasonsbehind selecting such a network was to use results from previousstudies [36] as a means of verifying the agent-based representation ofthe network system. Though the Braess network is a simplifiedversion of a networked system, consisting of a single road networkandmultiple bounded rational users, it preserves the characteristics oflarge complex network systems, namely, autonomous and boundedrational decision makers, user-network interactions and dynamic andemergent system equilibrium (traffic flow patterns). In particular,each individual user interacts with the rest of the system by playing a

Fig. 3. Netlogo screen shot for the ABM r

game in the form of making optimal travel choices based on theunderlying decision logic and the current network flow (Fig. 2).

In the first network depicted in Fig. 2, there are four nodes: (1, 2, 3,and 4), four links (a, b, c, and d), and a single O/D pair (1, 4). There are,hence, two paths available to travelers between this O/D pair: pathp1=(a, c) and path p2=(b, d). The user link travel cost functions are:

caðxaÞ = 10xa; cb xbð Þ = xb + 50; cc xcð Þ = xc + 50; cd xdð Þ = 10xd:

The Total system cost C is the summation over the set of all paths Pof user costs on each path p times the flow on that path, and is givenby:

C = Cp* fp;∀p∈ all paths P; fp is the total flow on path p ð1Þ

where C is the cost of travel for all users, and Cp is the cost incurred byall users on a path, i.e.

Cp = ∑i∈ℜ

ciðxiÞ*xi ð2Þ

where xi is the flow on link i and ci(xi) is the cost of travel over link i.Assume a fixed travel demand of 6. It is easy to verify that

the equilibrium path flows are: f114= f2

14=3. Note that thesuperscripts indicate the origin and destination nodes and thatthe subscripts indicate the path taken. The equilibrium link flows are:xa=xb=xc=xd=3, with associated equilibrium path travel costs:C114=C2

14=83.Assume now that, as depicted in Fig. 2, a new link “e”, joining node

2 to node 3 is added to the original network (thus giving rise to theBraess network), with user link cost function ce(xe)=xe+10. Theaddition of this link creates a new path p3=(a, e, d) that is availableto the travelers. Assume that the travel demand remains at 6 units of

epresentation of the Braess system.


flow. Note that the original flow distribution pattern f114=3, f214=3,

and f314=0 is no longer an equilibrium pattern, since at this level of

flow the user cost on path p3, C314=ca+cb+cc=70. Hence, usersfrom paths p1 and p2 would switch to path p3.

The equilibrium flow pattern on the new network is: f114=2,

f214=2, and f3

14=2. Hence the equilibrium link flows are: xa=4,xb=2, xc=2, xd=4, xe=2. The equilibrium user path travel costs are,C114=C2

14=C314=92. It can be verified that any reassignment of the

path flows would yield a higher travel cost on a path, and hence anunstable flow pattern. Note that the travel cost increased for everyuser of the network from 83 to 92 without a change in the totaldemand, and that the system performance diminished as a result ofthe addition of the link.1 Hence the total system cost, calculated as thesum of all link flows multiplied by all link travel times, would changefrom 498 to 552.

3.1. Policy design problem formulation for the Braess network

As seen in the previous section, the addition of a traveler choiceworsens the overall system cost. We now introduce a traffic flowcontrol mechanism at node 1 such that it will split the incoming trafficflow optimally between link a and link b, in order to reduce theoverall system cost. For example, this could be achieved by a timedsignal at the origin that stays on for a certain period of time for link aand b or by levying a toll on one of the links. To make the problemrealistic we introduce a few enhancements to the above Braessnetwork problem and present an optimization problem formulationfor the policy design scenario.

1. Instead of having a static demand (as in the case of the benchmarkproblem presented in the previous section), we consider a Poissonprocess driving the demand Q. Further the average occurrence ratein the Poisson process is modeled as a normal random variable.

2. The individual users using the network are considered to bebounded rational. They only look one link ahead and select thelowest cost link available.

3. The suggested policy, in order to decrease the overall system cost,is to introduce a flow control variable T at node 1. Given aninitial demand Q, (T/100 * Q) users are forced to travel on link aand (1−T/100) * Q users travel on link b. Users arriving at node 2behave in a bounded rational way as described above and selectthe lowest cost link between c and e.

4. The policy design problem is then to design an optimal value for Tsuch that the overall system cost for all users is minimized. Or inother words:

“Minimize the overall system cost with respect to the policy variableT such that the net flow on the network is conserved (flowconstraints) and the net flow on any link is greater than equal tozero”

This can be formally represented as a stochastic optimizationproblem with equilibrium constraints:

minT

EðCðTÞÞ

s:t: ∑pf rsp = Qrs ∀r∈ℜ; s∈S

Agent Route Choice Behavior

f rsp ≥0 ∀p∈P; r∈ℜ; s∈S:

1 The increased overall system cost due to addition of a traveler choice is popularlyreferred to as the Braess paradox.

4. Proposed policy design methodology

4.1. Agent-based representation of the Braess network

The first step in the policy design process as discussed previously isto build an agent-based representation of the underlying network.Weuse the agent-based approach for representing the Braess network.ABMs are very well suited for capturing multiple autonomousdecision makers displaying bounded rational behavior. We usedNetlogo [58] version 3.1 to build the agent-based representation ofthe Braess network (Fig. 3). Nodes 1–4 are modeled as Netlogopatches (fixed agents with attributes). Each patch contains informa-tion regarding its neighboring nodes and the current flow on each ofthe links between itself and its neighboring nodes. Each user ismodeled as a “turtle” (agents in Netlogo) that have a set of internalstate variables such as cost incurred and current position. One suchstate variable is start-link (referred to as T in the optimizationproblem formulation) that decides whether a user starts traveling onlink a or link b. As mentioned previously, depending on the value of T,(T/100 * Q) start on link a and (1−T/100) *Q on link b. The start-linkvariable is set with a simple mechanism; let's say if Q=20, andT=50/100 (50% will travel on link a and 50% on link b), then acounter is initiated that counts till T*Q and all agents born into thesystem have their start-link variable set to link a (after which rest ofthem have their start-link assigned to b). At each demand generationperiod all the agents are generated and once the “move” command isissued in parallel to all agents, they wait for a period of time (seconds)governed by a ∼U [0,3] before selecting a link based on the state oftheir internal start-link variable. This ensures that for every demandperiod T% split of the incoming demand occurs and the demand isloaded on to the network in a “random fashion”. The rules followed byeach user can be summarized as:

Rule 1: User selects between the links a and b based on the internalstate of their start-link variable.

Rule 2: An agent while deciding in a bounded rational way, alwaysconsiders only the current flow on the possible link optionsand selects the link immediately ahead with the least cost.

The agent-based model is then cast within the discrete timesimulation formalism [61]. At each simulation time instant a randomnumber of users (generated from the Poisson distribution mentionedearlier) are loaded at Node 1. This represents the demand faced by thesystem. Each user (after waiting for a time specified by their internalwait variable) starts traveling on a link (a or b) based on the value ofthe respective start-link variable. Further when these users eitherreach Node 2 or 3 they select a link that will take them to node 4(destination node). If a user reaches Node 3 it then proceeds on toNode 4 (only destination). A user who reaches Node 2 makes aselection between links c or e (using the bounded rational logic).Once a user selects a link it then travels on that link for a fixed time(constant parameter in the simulation model). Depending on whichlinks the users take the flows on the paths are calculated as follows.The flow on path p1 is the flow on link c. The flow on path p2 is flow onlink b and flow on path p3 is flow on link e. System cost is thencalculated based on Eqs. (1) and (2) and a running average isrecorded. Thus the data collected from the agent-based discretesimulation model of the Braess network system is recorded as timeseries data, which we utilize to calculate an expected value of thesystem cost. The expected value is then used further in surrogatemodel estimation and rest of the policy design process.

4.2. Model verification

Before proceeding to the stability analysis, response surfaceestimation and policy optimization stages, it is important to formally

Fig. 4. Latin hypercube sampling based design of experiment for Braess Networkproblem.


verify the ABM representation of the Braess network.We followed thefollowing verification steps:

1. We performed static and dynamic verification and testing assuggested by Fairley [18], and Sargent [49].1.1. For static testing we used structured walkthroughs through thesimulation code while considering a fixed demand scenario.1.2. For dynamic testing we used mixed traces as suggested byFairley [18]. We set up simulations with demands of 1 and 2. Wethen followed the movement of the individual agents (user of thenetwork) and ensured the decision logic is coded correctly.

2. We performed qualitative verification by simulating a traffic jamon link e. We accomplished this by increasing the travel time forany user on link e (10 times higher than users using link c).Although the links were not capacitated, this resulted in users toremain on that link for a long period of time as compared to link c.Thus after a critical mass was reached on link e any additional usercoming at Node 2 found link c to be cheaper than link e and startedchoosing link c. This step further ensured that the agents werecorrectly comparing the cost incurred to travel on a link.

3. We performed quantitative verification by testing the model withthe benchmark data from Nagurney and Dong [36], i.e., a staticdemand of 6 was used. The known result (from analyzing theequations) is an equal split between paths p1, p2 and p3 (flow of 2on each path). Without using the flow control variable Twe let theusers select links at nodes 1, 2 and 3 based on their boundedrational logic. Equilibrium flow of 2 users for each path wasobserved along with the total system cost of 552 (same as [36]).

Table 1Training data for response surface estimation on expected value of system cost for theBraess Network.

Experiment number Q T C

1 19 0.5 112217.82 14 0.83 26146.763 18 0.3 204055.74 11 0.17 53417.575 5 0.7 974.166 7 0.63 2261.9327 6 0.03 13593.988 13 0.1 131630.79 10 0.43 15234.6510 15 0.57 36224.4611 9 0.23 21852.1212 1 0.37 72.583

5. Analyzing stability of system response using Largest Lyapunovexponent characterization

For dynamic network systems that have multiple autonomousdecision makers and depict evolutionary behavior, stability analysis isan important step. Designing policies for scenarios where the systemshows unstable behavior may result in ineffective solutions. Dynamicsystems are typically said to exist in three possible states, i.e., stable,marginally stable and unstable (chaotic) [59]. Chaos or instability indynamic systems is identified by measuring the Largest Lyapunovexponent (LPE) [46]. Lyapunov exponents quantify the exponentialdivergence of initially close state-space trajectories and estimate theamount of chaos in a system [20,46,47,50]. In this paper we have usedthe algorithm and software (L1D2) developed by Rosenstein et al [46]to calculate LPE from the time series data (representing systemresponse over time) generated from the agent-based simulations.2

5.1. LPE calculation for the Braess system using L1D2 software

L1D2 takes time series data from the system as inputs andgenerates natural log of the divergence between neighboringtrajectories in the system at every point of the divergence time. LPEis evaluated by plotting the divergence data versus time to locate thelinear “scaling” region on the plot. The Largest Lyapunov exponent, ifit exists, is the slope over the scaling region. A positive value of LPEwould suggest chaos and instability, a value=0 depicts steady statebehavior. A valueb0 would depict a stable system. In case theresultant system behavior is unstable, the response surface should notbe estimated at those points and the policy maker has to try other setsof input variables for which the system would show stable behavior.For steady state behavior we have used a tolerance level of ±0.01 formarking when a system should be considered in a chaotic zone. Thetolerance level though is heuristically set is based on observations for

2 The text and description for the LPE algorithm has been borrowed from Rosensteinet al. (1993)(pp 4–6) and modified to meet this papers needs.

LPE values of known chaotic systems as mentioned in Rosenstein et al.[46].

In order to analyze the stability response of the agent-based modelrepresentation of the Braess network we designed 15 experimentswith different values of Q and T. The system cost was recorded fromeach of the 15 experiments. We used a Latin Hypercube Sampling-based design of experiment in order to efficiently sample the entirepolicy design space (we used the lhs function with correlationminimization in Matlab (version 7) to generate the design points asshown in Fig. 4). As mentioned previously Q was generated from aPoisson process with a normally distributed average occurrence rate.

Each experiment was executed on a cluster computing infrastruc-ture available at the Advanced Computing Center for Research andEducation (ACCRE) at Vanderbilt University and took 11 h of CPUtime. We used an average of the last 100 system cost data points asthe expected value of system cost from each of these experiments.Table 1 shows the results from the 15 experiments. LPE analysis wasthen performed using the L1D2 software as described previouslyusing the time series data on system cost from the 15 experiments.The LPE value came out to be 0 up to two decimal places for each of the15 experiments, indicating steady state behavior. Steady statebehavior in dynamic systems indicates that the system is operatingaround a fixed point. If the system is perturbed in the future it maydisplay more stable behavior (−ve LPE) or unstable behavior (+veLPE). Thus, systems in steady state display a weak notion of stabilityand any policy designed for such systems are effective only within theregion of operation defined by the input and control parameter space.Thus for the Braess network example, the designed policy can only betermed effective for the demand ranges where the system is stable.

13 2 0.77 154.735914 17 0.97 71074.7415 3 0.9 457.6457


6. Surrogate model estimation

The next step in the policy design process is to develop a surrogaterepresentation for the agent-based model in order to evaluate theobjective function that could be used by an optimization technique forfinding the exact value for T. There are numerous regression-basedsurrogate modeling techniques in the literature such as linearregression [16], non-linear regression [51], polynomial chaos models[23] and wavelet based models [41]. Linear, non-linear and polyno-mial chaos based regression techniques did not work well with thenon-linear and transient system responses observed in the Braesnetwork example. Though wavelet based regression methods havebeen used successfully for surrogate model estimation of systemswith non-linear response behavior with transients (stockmarket timeseries analysis, [39]), but they are computationally intensive andtypically the wavelet basis functions do not have a closed formrepresentation (except the Haar wavelets, which is not suitable forregression), thus making the task of formulating and solving theoptimization problems very hard and computationally intensive.

Gaussian process (GP) modeling is a powerful technique based onspatial statistics for interpolating data [43] that can be used to fitvirtually any functional form. GP models are increasingly being usedas surrogates to expensive computer simulations for the purposes ofoptimization and uncertainty propagation [32,33]. As a predictor, theGP model is non-parametric, meaning that no explicit functional formis assumed (e.g. linear, quadratic, etc.). In addition, the method can fitdata based on multiple input variables, and has been shown to beeffective for input dimensions as high as 30. Further, themethod givesa direct estimate as to the uncertainty associated with the value of theunknown function at any untested location. This uncertainty estimateis given in terms of a variance (a.k.a. mean squared error), and is afunction of the closeness of the new location to the existing datapoints.

6.1. Gaussian process surrogate model for the Braess network

The basic idea of the GP model is to model system response Y as agroup of multivariate normal random variables [43,48]. A parametriccovariance function is then constructed as a function of the systeminputs, x. The covariance function is based on the idea that when theinputs are close together, the correlation between the outputs will behigh. As a result, the uncertainty associated with the model'spredictions is small for input values which are close to the trainingpoints, and large for input values which are not close to the trainingpoints. In addition, the mean function of the GP may capture large-scale variations, such as a linear or quadratic regression of the inputs.

Table 2Training data for response surface estimation on expected average speed in the trafficgrid problem.

Experiment number Number of cars Speed limit Ticks per cycle Average speed

1 140 0.17 31 0.1162 113 0.30 11 0.2673 7 0.83 21 0.434 127 0.70 15 0.315 73 0.90 17 0.4326 20 0.50 37 0.3017 47 0.03 29 0.0648 60 0.77 23 0.3889 180 0.97 39 0.03410 153 0.43 13 0.23211 33 0.57 27 0.30812 193 0.63 25 0.19413 167 0.23 33 0.13814 87 0.37 35 0.21215 100 0.10 19 0.111

Thus Y can be represented by a Gaussian process with mean β andcovariance function defined as:

E½YðxÞ� = β ð3Þ

and

Cov½YðxÞ; Yðx*Þ� = λcðx; x* jϕÞ ð4Þ

where c(x, x*|ϕ) is the correlation between x and x*, ϕ is the vector ofparameters governing the correlation function, and λ is the processvariance. Following are the steps for estimating the surrogaterepresentation:

1. Simulate system at m training points to generate system responsebehavior:

Y = ðYðx1Þ;…; YðxmÞÞT ð5Þ

In the Braess example m=15 training points selected using theLatin Hypercube Sampling-based design process. Y representssystem cost C as given by Eq. (1).

2. Construct a joint distribution of Y given by

Y∼Nmðβ1;λRÞwhereR isam*mmatrixof coorrelationsamongstthetrainingpoints:

ð6Þ

3. The expected value and variance at any input x are then given by

E½YðxÞ� = β + rT ðxÞR−1ðY−β1Þ ð7Þ

and

Var½YðxÞ� = λð1−rTR−1rÞ ð8Þ

where r is the vector of correlations between x and each of thetraining points. x is the vector of Q and T in the Braess problemdomain. Please note that T (italicized) in the above Eqs. (7) and(8) refers to transpose of a matrix rather than our design variableT (non-italicized).

Using training data presented in Table 1 and the steps describedabove, we built a GP model estimate for the Braess network system.The GP model building scheme was developed and implemented inMATLAB [33] and was used for building the surrogate model shown inFig. 5.

Fig. 5. Plot of response surface using the GP surrogate model for Braess Networksystem.

Fig. 7. Traffic grid.


7. Stochastic optimization of policy variable

As seen in Fig. 5, the surrogate model plot for the Braess networkshows how the overall system cost worsens for low values of T andincreased demand conditions. In order to determine an exact value forT we suggest a stochastic optimization problem formulation (sincedemand is stochastic) and solution using golden section search [56].We decided to use such a simple approach as the visual plot of theresponse surface clearly shows the region around which we couldexpect to find the optimal value for T. Thus the stochastic optimizationproblem is to minimize the expected value of the system cost withrespect to T, and subject to the stochastic demand (Q) and the GPmodel. It can be formally stated as:

MinT

E½C�s:t:Q≥0E½YðxÞ� = β + rT ðxÞR−1ðY−β1Þ

where r, R and β are described by Eqs. (3), (6)–(8).We use MATLAB's fminbnd golden section search algorithm to

minimize the expectation of the system cost with respect to thedecision variable T. The algorithm minimizes the expected value ofthe system cost, which is calculated in the manner described by thefollowing MATLAB code. The function expected_cost accepts T and alist of common random numbers (to aid by elimination of sample-to-sample variance in estimation of the objective function) as anargument, and returns the expected system cost. The MATLAB codefor this function is:

function E=expected_cost(T, common_random_numbers)

for i=1 to length(common_random_numbers)

system_costs(i) =cost_function_GP( T,common_random_numbers(i,:));i= i+1;

end

E=sum(system_costs)/i;

The expected cost function is optimized with the fminbnd solverroutine.

Although golden section search does not guarantee global optima,in this case with the help of the response surface plot we were able toconclude decisively that T=0.73 is indeed the optimal value (Figs. 6).

8. Scalability demonstration

In order to address the issue of scalability we have solved aproblem involving optimal control of traffic in a grid network of the

Fig. 6. Results of golden section search algorithm for Braess Network system.

type often found in urban central business districts (Fig. 7). Our gridproblem involves 25 signalized intersections with identical cycles, inphase with each other, and a cycle specified by the modeler (fromhere on referred to as ticks per cycle). Such signal timing strategies arealso encountered in the central business districts with uniform cityblocks (Manhattan, Seattle, Chicago etc). Our grid has 25 nodes, 80links, and approximately an order of magnitude more agents (tentimes as compared to the Braess example).

8.1. Model building and data collection

We use a streamlined traffic grid model developed by Wilenskiet al. in Netlogo [43]. The model uses software agents (as in the Braessexample) for modeling individual users. As an initial condition of thesimulation we position the cars randomly over the grid. Once thesimulation begins the agents follow three simple rules:

Rule 1: Each time step, the cars attempt to move forward at theircurrent speed. If their current speed is less than the speedlimit and there is no car directly in front of them, theyaccelerate.

Rule 2: If there is a slower car in front of them, they match the speedof the slower car and decelerate.

Rule 3: If there is a red light or a stopped car in front of them, theystop.

The grid is shaped in the form of a torroid. In other words, the carsreappear at the beginning of a street after traversing the entire street.This simulates a constant population over the simulation time period.There can be number of implementations representing differentscenarios, such as if new cars entered the grid at random locations andintervals. We chose the scenario where we are investigating aconstant demand during a certain period of the day such as rushhour. We treat demand, i.e., the number of cars using the grid, as arandom variable. The objective is to improve operating conditions(the expectation of the average speed over the network) bymanipulating two variables, the cycle length (represented by ticksper cycle) for the red and green signals and the speed limit (set to 1.0).The expected travel speed over the network is a nonlinear and convexfunction of the cycle length, number of cars, and speed limit.

Max E½Average SpeedðCycle Length; Speed LimitÞ� ð9Þ

Fig. 8. Plot of response surface using the GP surrogate model for traffic grid.


where the expectation (E) is calculated by Monte-Carlo simulation ofthe Gaussian Process surrogate model.

This problemwas solved with only 15 agent-based simulations (attraining points selected using a Latin Hypercube Sampling mecha-nism) (Table 2). Each agent-based simulation took approximately 40 sto run on a computer with 1 GB RAM with 5000 time steps. Asmentioned above the simulation starts with a random placement ofcars in the grid and we repeated each training point (inputs) a 1000times finally taking the average value of the output (average speed ofcars) as the training data (output). Thus it took 40,000 s (∼11 h) forcollecting data at each training point.

8.2. Stability analysis

We perform a stability analysis and calculate the Largest LyapunovExponent for each training point using the Rosenstein algorithm [46]and L1D2 software as mentioned previously in the Braess exampleand ensured that the training data used for surrogate model buildingrepresent stable system states. We found the LPE values to be withinthe tolerance limit of (±0.01). Hence as in the case of the Braessexample, we found the traffic grid system to exhibit weakly stablebehavior (steady state) and the prescribed policy designs would beapplicable within the range of the input variables (Number of cars,Speed Limit and Ticks per cycle).

8.3. Surrogate estimation and optimization

Next we proceeded to build a GP response surface using thetraining data and followed it up with extended prediction testing. Wecompared the GP model and simulation results at points other than

Fig. 9. Plot of stability response

the training points and found the surrogate to be highly accurate(within ±3% of the simulation runs). At this point we performed agrid search over the response surface involving 90 trial designs (policyscenarios), with 10,000 samples taken for each trial design to evaluatethe average speed per scenario combinations. Recall that, randomnessis introduced in the simulation by placing the initial start points of thecars on the grid in a random fashion. Using the surrogate model, thisoptimization was completed in 204 s. Such an approach would havebeen impossible to carry out with an expensive agent basedsimulation (∼1130 years). The optimal policy variable was found bysimply taking the MAX of the average speed values obtained. Theroutine was implemented in MATLAB. The response surface and theMAX value is shown in Fig. 8.

From Fig. 9, optimal policy is to set the ticks per cycle at 20 and thespeed limit at 1. An interesting question at this stage is whether theoptimal solution is the most stable solution too? In order to addressthis question, we utilized the GP model technique (as per the processdescribed previously) to build a stability response surface using theLyapunov values calculated at the training points (and using cubicinterpolation to build a response surface in MATLAB, as shown inFig. 9). Fig. 9 shows that the optimal policy of 20 ticks per cycle andspeed limit of 1 is not the most stable configuration. In fact setting thepolicy to 44 ticks per cycle and a speed limit between 0.3 and 1 resultsin the lowest Lyapunov exponent (−ve), indicating stability for thesystem but at the cost of reduced average speed. Thus approaching thepolicy design problem from two different angles we get two differentanswers. One solution yields higher average speed (0.375) butoperates in a weekly stable zone (higher variance), while the othersolution yields a lower average speed (max 0.21 for a speed limit of 1)but operates in a stable zone (lower variance as compared to the 20ticks per cycle scenario). Based on our experience with the model wesaw that once the zones of operation has been identified fine grainsimulation runs can be used for scenario generations. Thus using thisframework, policy designer can make structured and informeddecisions based on their priorities.

9. Summary and future work

In this paper, we presented a framework to design stable policiesfor network systems by synthesizing agent-based modeling, stabilityanalysis of dynamic systems, surrogate estimation using GP modelingand optimization under uncertainty (OUU) within an over-archingframework. We illustrated the framework using a popular benchmarkproblem from the transportation network literature, i.e., using Braessnetwork [6]. Additionally we considered users with bounded rationalbehavior. We modeled individual users as software agents [21] inNetlogo [58] following two simple rules. The agent-based model(ABM)was implemented as a discrete time simulation, where at everysimulation time instant, demand was loaded on to the system. We

surface for the traffic grid.


used Largest Lyapunov exponent (LPE) characterization of the systemresponse behavior to identify the stability/instability/steady statecharacteristics of the network system for a given set of input andcontrol parameters. Further, the paper demonstrated how a Gaussianprocess (GP) model [43,48] can be utilized to develop an inexpensivesurrogate for the ABM model thus facilitating the optimization of thepolicy variable using stochastic optimization methods. The paperintroduces the construction of stability response surfaces usingLyapunov values at a very limited set of training points and using itfor trade-off analysis. The framework is completely agnostic to theunderlying modeling methodology as it only assumes the presence ofa limited set of training point data. Lastly, the framework presented inthis paper is general and suitable for solving not only transportationproblems, but also other problems such as supply networks, financialnetworks and combined or super networks.

While we feel that the framework is a step forward towardsaddressing the challenges of policy design for networked systems, ithas certain limitations. Designing policies for networked systems iseffectively an optimal control problem [36]. Optimal control problemscome in two varieties, open loop and closed loop. Open loop controlsystems use predetermined policies. Real-time information about thecurrent state of the network is not used to modify the control. Suchopen loop strategies are popularly used in transportation networkresearch [53]. In principle open loop control problems can be solvedby discretizing control variables over time and using the optimizer tomanipulate these vectors. Thus the solutions generated hold true overa range of the input and policy variables (thus the static nature). Amore sophisticated approach would be to use closed loop feedbackduring the optimization process, simultaneously as the systemevolves. This framework does not lend itself towards solving closedloop control problems and further research is needed that seems to bebeyond the scope of just one paper.

In future research, we will apply the current framework to solvepolicy design problems in multiple class, multiple mode supernetworks as suggested by Nagurney et al. [37]. Also systemdynamics [22] and cellular automaton [60] modeling can be usedin place of ABMs. One of the most important characteristics of real-world network-systems problem is the presence of aleatoryuncertainties such as randomness in the demand function, as wellas epistemic uncertainties such as imperfect information, at variousdecision making levels. In this paper we have shown how aleatoryuncertainties in the network system can be handled. Futureresearch needs to investigate how epistemic uncertainties at theinformation level as well at the model level (lack of completeinformation about the system itself) could be specifically consid-ered while designing policies. For example, policy designers maylack knowledge of exact user behavior while users may lackknowledge of the operational state of the system. The agent-basedapproach could potentially be used to model intelligent andadaptive decision making behavior at the individual user level inorder to address the uncertainty arising due to the lack of specificknowledge. The exact method of modeling such mechanisms inABMs and their impacts on network policy design needs to beinvestigated future research.

Acknowledgments

We thank the IGERT program and Advanced Computing Center forResearch at Vanderbilt University for supporting our research. Ad-ditionally we thank John McFarland at the South Western ResearchLaboratories for help with the Gaussian Process modeling in MATLABand Dr. Michael T. Rosenstein, Senior Research Scientist at theDepartment of Computer Science, University of Massachusetts, Amherstfor valuable suggestions and help with the Lyapunov exponentcomputation.

References

[1] M.C. Agar, S. Salhi, Lagrangean heuristics applied to a variety of large capacitatedplant location problems, Journal of the Operational Research Society 49 (1998)1072–1084.

[2] K.S. Al-Sultan, M.A. Al-Fawzan, A tabu search approach to the uncapacitatedfacility location problem, Annals of Operations Research 86 (1999) 91–103.

[3] R.G. Aldunate, F. Pena-Mora, G.E. Robinson, Collaborative distributed decisionmaking for large scale disaster relief operations: drawing analogies from robustnatural systems, Complexity 11 (2) (2005).

[4] O. Alp, E. Erkut, Z. Drezner, An efficient genetic algorithm for the p-medianproblem, Annals of Operations Research 122 (2003) 21–42.

[5] R. Axelrod, The complexity of cooperation: agent-based models of competitionand collaboration, Princeton University Press, 1997.

[6] D. Braess, Uber ein Paradoxon der Verkehrsplanung, Unternehmensforschung12 (1968) 258–268.

[7] K.M. Carley, Smart agents and organizations of the future, in: L.L.a.S. Livingstone(Ed.), The Handbook of New Media, Sage, Thousand Oaks, CA, 2002.

[8] K.M. Carley, N. Kamneva, A network optimization approach for organizationaldesign, Carnegie Melon Uni\versity, Pittsburgh, 2004.

[9] G.L. Chang, H.S. Mahmassani, Travel time prediction and departure time adjustmentbehavior dynamics in a congested traffic system, Transportation Research B 22 (3)(1988) 217–232.

[10] T.Y. Choi, K.J. Dooley, M. Rungtusanatham, Supply networks and complex adaptivesystems: control versus emergence, Journal of Operations Management 19 (3)(2001) 351–366.

[11] S. Chopra, P. Meindl, Supply chain management: strategy, planning, operation,2nd ed.Prentice Hall, 2003.

[12] M.G. Cojocaru, P. Daniele, A. Nagurney, Projected dynamical systems, evolutionaryvariational inequalities, applications, and a computational procedure, in: A.M.A.Chichuluun, P.M. Pardalos, L. Pitsoulis (Eds.), Pareto Optimality, Game Theory andEquilibria, Springer, Berlin, Germany, 2007.

[13] C.J. Daganzo, J. Laval, J.C. Munoz, Some ideas for freeway congestion mitigationwith advanced technologies, Traffic Engineering Control 43 (10) (2002) 397–403.

[14] M.S. Daskin, Network and discrete location: models, algorithms, and applications,J Wiley and Sons, Inc, New York, 1995.

[15] D.A. DeLaurentis, R.K. Callaway, A system-of systems perspective for future publicpolicy, Review of Policy Research 21 (6) (2004) 829–837.

[16] N.R. Draper, H. Smith, Applied regression analysis, Wiley Series in Probability andStatistics, Wiley, 1998.

[17] H.A. Eiselt, C.L. Sandblom, Decision analysis, location models, and schedulingproblems, Springer-Verlag, Berlin, 2004.

[18] R.E. Fairley, Dynamic testing of simulation software, Proceedings of the 1976Summer Computer Simulation Conference, Washington D.C., 1976.

[19] J. Farley, in: M. Loukides (Ed.), Java Distributed Computing, 1st ed., O'Reilly Media,Sebastopol, CA, 1998.

[20] J.D. Farmer, J.J. Sidorowich, Predicting chaotic time series, Physics Review Letters59 (1987) 845.

[21] J. Ferber, Multi-agent systems: an introduction to distributed artificial intelligence,Addison-Wesley, Harlow, England, Don Mills, Ont., 1999

[22] J.W. Forrester, Industrial dynamics, MIT Press, Cambridge, MA, 1961.[23] R.G. Ghanem, P.D. Spanos, Stochastic finite elements: a spectral approach,

Springer-Verlag, 1991.[24] G. Guariso, M. Hitz, H. Werthner, An integrated simulation and optimization

modeling, environment for decision support, Decision Support Systems 16 (1996)103–117.

[25] J.N. Hagstrom, R.A. Abrams, Characterizing Braess's paradox for traffic networks,IEEE Intelligent Transportation Systems Conference Proceedings, Oakland (CA)USA, IEEE, 2001.

[26] C.M. Harland, R.C. Lamming, J. Zheng, T.E. Johnsen, A taxonomy of supplynetworks, Journal of Supply Chain Management 37 (4) (2001) 21–28.

[27] Y.C. Ho, X.R. Cao, Perturbation analysis of discrete event dynamics, Perturbationanalysis of discrete event dynamics, 1991.

[28] M. Hribar, M.S. Daskin, A dynamic programming heuristic for the p-medianproblem, European Journal of Operational Research 101 (1997) 499–508.

[29] N. Kang, S. Han, Agent-based e-marketplace system for more fair and efficienttransaction, Decision Support Systems 34 (2002) 157–165.

[30] R. Klashner, S. Sabet, A DSS design model for complex problems: lessons frommission critical infrastructure, Decision Support Systems 43 (2007) 990–1013.

[31] B. Lewis, D.J. Berg, Threads primer: a guide to multithreaded programming,Prentice Hall, Upper Saddle River, NJ, 1995.

[32] J.D. Martin, T.W. Simpson, A study on the use of kriging models to approximatedeterministic computer models, Proceedings of DETC'03 ASME 2003 DesignEngineering Technical Conferences and Computers and Information in EngineeringConference, Chicago, Illinois USA, 2003.

[33] J. McFarland, S. Mahadevan, V. Romero, L. Swiler, Calibration and uncertaintyanalysis for expensive computer simulations with multivariate output, Proceed-ings of the 9th AIAA Non-Deterministic Approaches Conference, Schaumburg, IL,2008.

[34] L.W. Miller, N. Katz, A model management system to support policy analysis,Decision Support Systems 2 (1) (1986) 55–63.

[35] P.K.J. Mohapatra, S.K. Sharma, Synthetic design of policy decisions in systemdynamics models: a modal control theoretical approach, System DynamicsReview 1 (1) (1985) 63–80.

[36] A. Nagurney, J. Dong, Supernetworks: decision-making for the information age,Edward Elgar Publishers, Cheltenham, England, 2001.

S. Pathak et al. / Decision Support

[37] A.D. Nagurney, D. Parkes, P. Danniele, The internet, evolutionary variationalinequalities, and the time-dependent Braesss paradox, Computational ManagementScience 4 (2007) 243–281.

[38] J.F. Nash, Equilibrium points in n-person games, Proceedings of National Academyof Science 36 (1950) 48–49.

[39] Z. Pan, X. Wang, A stochastic non-linear regression estimator using wavelets,Computational Economics 11 (1998) 89–102.

[40] S.D. Pathak, D.M. Dilts, G. Biswas, On the evolutionary dynamics of supplynetwork topologies, IEEE Transactions on EngineeringManagement 54 (4) (2007)662–672.

[41] D.B. Percival, A.T. Walden, Wavelet methods for time series analysis, CambridgeUniversity Press, Cambridge, 2000.

[42] Q. Qiang, A. Nagurney, A unified network performance measure with importanceidentification and the ranking of network components, Optimization Letters 2 (1)(2008) 313–426.

[43] C. Rasmussen, Evaluation of Gaussian processes and other methods for non-linearregression, University of Toronto, Toronto, 1996.

[44] C.S. ReVelle, H.A. Eiselt, Location analysis: a synthesis and survey, EuropeanJournal of Operational Research 165 (1) (2005) 1–19.

[45] C.W. Reynolds, Flocks, herds, and schools: a distributed behavioral model, incomputer graphics, SIGGRAPH '87 Conference Proceedings, 1987.

[46] M.T. Rosenstein, J.J. Collins, C.J.D. Luca, A practical method for calculating largestLyapunov exponents from small data sets, Physica D 65 (1993) 117–134.

[47] M. Sano, Y. Sawada, Measurement of the Lyapunov spectrum from a chaotic timeseries, Physics Review Letters 55 (1985).

[48] T.J. Santner, B.J. Williams, W.I. Noltz, The design and analysis of computerexperiments, Springer-Verlag, New York, 2003.

[49] R.G. Sargent, Verifying and validating simulation models, Proceedings of 1996Winter Simulation Conference, 1996.

[50] S. Sato, M. Sano, Y. Sawada, Practical methods of measuring the generalizeddimension and the largest Lyapunov exponent in high dimensional chaoticsystems, Progress of Theoretical Physics 77 (1987).

[51] G.A.F. Seber, C.J. Wild, Nonlinear regression, JohnWiley and Sons, New York, 1989.[52] C.R. Shalizi, Methods and techniques of complex systems science: an overview, in:

T.S. Deisboeck, J.Y. Kresh (Eds.), Complex Systems Science in Biomedicine,Springer, New York, 2006.

[53] Y. Sheffi, Urban transportation networks: equilibrium analysis with mathematicalprogramming methods, Prentice-Hall, Englewood Cliffs, New Jersey, 1985.

[54] H.A. Simon, Models of man: social and national, Wiley, New York, 1957.[55] S.C. Dafermos, F.T. Sparrow, The traffic assignment problem for a general network,

Journal of Research of the National Bureau of Standards 73 (B) (1969) 91–118.[56] R.J. Stuart, N. Peter, Artificial intelligence: a modern approach, 2nd ed.Prentice

Hall, Upper Saddle River, N.J., 2003[57] T. Sueyoshi, G.R. Tadiparthi, An agent-based decision support system for

wholesale electricity market, Decision Support Systems 44 (2008) 425–446.[58] U. Wilensky, NetLogo, Center for Connected Learning and Computer-Based

Modeling, Northwestern University, Evanston, IL, 1999.[59] G.P. Williams, Chaos theory tamed, vol. xvii, Joseph Henry Press, Washington, D.C.,

1997, p. 499.[60] S. Wolfram, A new kind of science Wolfram Media, , 2002.[61] B.P. Zeigler, H. Praehofer, T.G. Kim, 2nd ed., Theory of modeling and simulation:

integrating discrete event and continuous complex dynamic systems, vol. xxi,Academic, San Diego, California, 2000, p. 510.

f

,

t

,

l

l.

,

,

,

,,

Dr. Surya Pathak is an Assistant Professor of OperationsManagement in the Business Program at University oWashington Bothell. He received his PhD in InterdisciplinaryManagement of Technology from Vanderbilt University in2005.He is currently conducting research in theareaofpolicydesign for large-scale systems, complex adaptive supply

131Systems 49 (2010) 121–131

networks, decision making under risk and uncertaintysupply network design, and supply relationship manage-ment. His methodological orientations include agent-basedsimulations and cellular automaton models on grid comput-ing infrastructure alongwithmathematicalmodeling, robusand reliability-based design optimization, archival dataanalysis, and game theoretic modeling techniques for

investigating policy implications in diverse domains, such as manufacturing and healthcare supply networks, transportation networks, and super networks. He is a foundingmember of the Center for Supply Network Science at Arizona State University, at TempeArizona. Dr. Pathak's work has been published or is under consideration in the IEEETransactions on Engineering Management, Journal of Operations Management, InternationaJournal of Production Research, Decision Support Systems, Journal of Supply Chain Manage-ment, Decision Sciences and Transportation Research Records.

Dr. Mark Philip McDonald is an Assistant Professor of Civiand Environmental Engineering at Vanderbilt UniversityHe has earned the BCE from Auburn University in 2003, theMSCE from the University of California, Berkeley in 2004and the PhD from Vanderbilt University in 2008. Hisresearch interests include optimal design, uncertainty
analysis, and decision making under uncertainty forcomplex systems, particularly systems of systems.
Dr. Sankaran Mahadevan is Professor of Civil and Environ-mental Engineering, and Director of Vanderbilt University'sNSF-funded multidisciplinary graduate program in Reliabil-ity and Risk Engineering and Management. He received hisPhD in 1988 fromGeorgia Institute of Technology. Hisresearch contributions are in model-based simulation
mechanical systems durability, reliability and risk assess-ment, design optimization and model validation techniqueswith applications to civil infrastructure, automotive andaerospace systems. His research has been funded by NSFNASA, DOD, DOE, FAA, GM, DaimlerChrysler, Union Pacificand Oak Ridge, Sandia and Idaho National Laboratories. Hisresearch is documented in more than 300 publications.

Documents

A framework for designing policies for networked systems with uncertainty