Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Distributed Parallel Process Particle SwarmOptimization on Fixed Charge Network
Flow Problems
Corey Clark1 Charles Nicholson2
1Game Theory Labs, Dallas, [email protected]
2University of Oklahoma, Industrial and Systems Engineering, Norman, [email protected]
INFORMS Annual Meeting, 2013
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Outline
1 IntroductionThe ProblemThe Model
2 Problem ApproachesOptimal SearchHeuristic Search
3 Dynamically Distributed BPSO ApproachAlgorithmArchitecture
4 Performance Results and DemonstrationPerformance ResultsDemonstration
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
The Problem
Problem Motivation: Cash Management2
National banks managevaults that store cashVaults have excess ordeficit (current / forecast)Routing cash incursfixed and variable costsModeled as Time-spacefixed-charge networkflow problem1
1J. Kennington and C. Nicholson. “The Uncapacitated Time-Space Fixed-ChargeNetwork Flow Problem: An Empirical Investigation of Procedures for Arc CapacityAssignment”. In: INFORMS Journal on Computing 22 (2010), pp. 326–337.
2M. Frost, J. Kennington, and A. Madhavan. “Optimizing cash management for largescale bank operations”. In: International Journal of Operations Research andInformation Systems 1 (2010), pp. 17–31.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
The Model
Time-Space Fixed-Charge Network Flow Model
Graph
N is set of n spatial nodes; T is the set of t time periods, and N̄is the set of node-time pairsA is the set of arcs: (i , r , j , s) where (i , r) ∈ N̄, (j , s) ∈ N̄
Variablesxirjs is flow on arc (i , r , j , s) ∈ A; yirjs is related binary variable
ParametersMirjs is an implied artificial arc capacity used in B&B modelingcirjs and firjs are variable and fixed costs for arc (i , r , j , s)Rir are the requirements at node (i , r) ∈ N̄.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
The Model
Time-Space Fixed-Charge Network Flow Model
Graph
N is set of n spatial nodes; T is the set of t time periods, and N̄is the set of node-time pairsA is the set of arcs: (i , r , j , s) where (i , r) ∈ N̄, (j , s) ∈ N̄
Variablesxirjs is flow on arc (i , r , j , s) ∈ A; yirjs is related binary variable
ParametersMirjs is an implied artificial arc capacity used in B&B modelingcirjs and firjs are variable and fixed costs for arc (i , r , j , s)Rir are the requirements at node (i , r) ∈ N̄.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
The Model
Time-Space Fixed-Charge Network Flow Model
Graph
N is set of n spatial nodes; T is the set of t time periods, and N̄is the set of node-time pairsA is the set of arcs: (i , r , j , s) where (i , r) ∈ N̄, (j , s) ∈ N̄
Variablesxirjs is flow on arc (i , r , j , s) ∈ A; yirjs is related binary variable
ParametersMirjs is an implied artificial arc capacity used in B&B modelingcirjs and firjs are variable and fixed costs for arc (i , r , j , s)Rir are the requirements at node (i , r) ∈ N̄.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
The Model
TSFC Problem Formulation
Given a directed graph G = (N̄,A) the time-space fixed-chargenetwork flow model is formally stated as follows:
TSFC Problem
min∑
(i,r ,j,s)∈A
(cirjsxirjs + firjsyirjs) (1)
s.t.∑
(i,r ,j,s)∈A
xirjs −∑
(j,s,i,r)∈A
xjsir = Rir ∀(i , r) ∈ N̄ (2)
0 ≤ xirjs ≤ Mirjsyirjs ∀(i , r , j , s) ∈ A (3)yirjs ∈ {0,1} ∀(i , r , j , s) ∈ A (4)
This problem is NP-hard.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Optimal Search
Optimization Approach
Branch-and-Bound for the TSFC problem1 Problem is solved with relaxed binary constraints2 Choose an arc (i , r , j , s) ∈ A for branching
create new subproblem with yirjs = 1create new subproblem with yirjs = 0
3 Solve the relaxed sub-problems (linear programs)4 Use results to update bounds, determine optimality, fathom
nodes, and continued branching5 Go to step 2
B&B performs a complete (implicit) search among all possiblenetwork designs (branches) to find an optimal solution.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Optimal Search
Search Space
The set of all feasible solutionsis called a search spaceEach point in the spacerepresents one feasiblesolutionEvery point has an associatedfitness valueThe set of solutions and theirobjective values form locationsand elevation in the searchspace landscapeThe search space can belarge and complex
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Heuristic Search
Heuristic Approach
LiteratureTabu search (Glover 1990)
Fixed-Charge Transportation Problem (Sun et al. 1998)Genetic Algorithms (Holland 1975)
Non-linear Transportation Problem (Sheng et al. 2006)Fixed-Charge Network Flow (Duhamel 2010)Network Designs (Gen and Chang 2003)
Particle Swarm Optimization (PSO)Real-valued PSO (Kennedy and Eberhart 1995)Binary PSO (Kennedy and Eberhart 1997)Many applications and enhancements since then, e.g.Parsopoulos and Vrahatis 2007, Yin et. al 2010
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Algorithm
Binary Particle Swarm Optimization
Particle position and velocityXi = (xi1, xi2, . . . , xik ) xij ∈ {0,1} for j = 1,2, . . . , kVi = (vi1, vi2, . . . , vik ) vii ∈ [−vmax , vmax ] for j = 1,2, . . . , k
The position Xi represents a solution, and has anassociated fitness valueA swarm of n particles fly through the k -dimensionalsolution space – updating their positions based on avelocity functionVelocity is a function of the historical best solution foundindividually by a particle and globally by the swarmThe velocity is converted into a probability that is used tomodify the particle position
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Algorithm
Binary Particle Swarm Optimization
Particle position and velocityXi = (xi1, xi2, . . . , xik ) xij ∈ {0,1} for j = 1,2, . . . , kVi = (vi1, vi2, . . . , vik ) vii ∈ [−vmax , vmax ] for j = 1,2, . . . , k
The position Xi represents a solution, and has anassociated fitness value
A swarm of n particles fly through the k -dimensionalsolution space – updating their positions based on avelocity functionVelocity is a function of the historical best solution foundindividually by a particle and globally by the swarmThe velocity is converted into a probability that is used tomodify the particle position
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Algorithm
Binary Particle Swarm Optimization
Particle position and velocityXi = (xi1, xi2, . . . , xik ) xij ∈ {0,1} for j = 1,2, . . . , kVi = (vi1, vi2, . . . , vik ) vii ∈ [−vmax , vmax ] for j = 1,2, . . . , k
The position Xi represents a solution, and has anassociated fitness valueA swarm of n particles fly through the k -dimensionalsolution space – updating their positions based on avelocity function
Velocity is a function of the historical best solution foundindividually by a particle and globally by the swarmThe velocity is converted into a probability that is used tomodify the particle position
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Algorithm
Binary Particle Swarm Optimization
Particle position and velocityXi = (xi1, xi2, . . . , xik ) xij ∈ {0,1} for j = 1,2, . . . , kVi = (vi1, vi2, . . . , vik ) vii ∈ [−vmax , vmax ] for j = 1,2, . . . , k
The position Xi represents a solution, and has anassociated fitness valueA swarm of n particles fly through the k -dimensionalsolution space – updating their positions based on avelocity functionVelocity is a function of the historical best solution foundindividually by a particle and globally by the swarm
The velocity is converted into a probability that is used tomodify the particle position
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Algorithm
Binary Particle Swarm Optimization
Particle position and velocityXi = (xi1, xi2, . . . , xik ) xij ∈ {0,1} for j = 1,2, . . . , kVi = (vi1, vi2, . . . , vik ) vii ∈ [−vmax , vmax ] for j = 1,2, . . . , k
The position Xi represents a solution, and has anassociated fitness valueA swarm of n particles fly through the k -dimensionalsolution space – updating their positions based on avelocity functionVelocity is a function of the historical best solution foundindividually by a particle and globally by the swarmThe velocity is converted into a probability that is used tomodify the particle position
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Algorithm
Binary Particle Swarm Optimization Updates
Let Pi denote the best position visited by particle iLet Pg denote the best position found by the swarm
The velocity function at iteration t + 1 is:
V t+1i = wV t
i + C1r1(Pi − X t
i)
+ C2r2(Pg − X t
i)
where w is an inertia factor that changes over time; C1,C2 areconstants; r1, r2 are random values uniform on [0,1]
The transfer function that converts velocities to probabilities
T (vij) =1
1 + evij
is used to update each bit the position according to:
xij =
{1, if U(0,1) < T (vij)
0, otherwise
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Algorithm
Binary PSO and the FCNF Problem
In our application of Binary PSO tothe Fixed-Charge Network FlowProblem, each particle positionrepresents a network design.
Each position is a unique, purenetwork problem which is solved todetermine the minimum cost of theparticular design.
The individual designs are entirelyindependent of each other, whichmakes this easy to parallelize.
X1 :
X2 :
X3 :
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Architecture
Distributed Accelerated Analytics Platform
Distributed Accelerated Analytics Platform (DAAP)DAAP enables dynamic creation of a heterogeneous clusternetwork using HTML5 technologies that provide multi-threadedexecution and low latency connections.
GLPK and GLPK.jsThe GNU Linear Programming Kit (GLPK) package solveslarge-scale LP and MIP problems.GLPK.js: JavaScript interfaces to GLPK which allow GLPK tobe called from webpages on either the client side or the serverside.3
Any web-enabled device can become part of a DAAP clusternetwork and contribute computing power to solve LP problems.
3H. Gourvest. GLPK.js. URL: https://github.com/hgourvest/glpk.js.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Architecture
Distributed Accelerated Analytics Platform
Distributed Accelerated Analytics Platform (DAAP)DAAP enables dynamic creation of a heterogeneous clusternetwork using HTML5 technologies that provide multi-threadedexecution and low latency connections.
GLPK and GLPK.jsThe GNU Linear Programming Kit (GLPK) package solveslarge-scale LP and MIP problems.GLPK.js: JavaScript interfaces to GLPK which allow GLPK tobe called from webpages on either the client side or the serverside.3
Any web-enabled device can become part of a DAAP clusternetwork and contribute computing power to solve LP problems.
3Gourvest, GLPK.js.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Performance Results
Empirical Testing: Problem Characteristics
We tested our approach on three different sized TSFCProblems. Feasible instances were randomly generated andthe variable and fixed cost ranges were selected such as tocreate difficult instances (Kennington and Nicholson 2010).
Problems
Variable Costs: U(0,10)Fixed Costs: U(20000,60000)
Size Nodes Arcs5n6p 30 24510n21p 210 3,89020n30p 600 23,000
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Performance Results
Empirical Testing: Approaches
Three approaches were tested:Optimization: GLPK.js with default MIP techniques
Serial BPSO-FCNFDistributed BPSO-FCNF
Devices used in the DAAP cluster include: MacBookPro,Surface Pro, iPhone, Windows laptop, iPad, Nexus 7tablet
A time limit was set at 10 minutes and best objective valuesand number of network problems solved are reported.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Performance Results
Empirical Testing: Approaches
Three approaches were tested:Optimization: GLPK.js with default MIP techniquesSerial BPSO-FCNF
Distributed BPSO-FCNFDevices used in the DAAP cluster include: MacBookPro,Surface Pro, iPhone, Windows laptop, iPad, Nexus 7tablet
A time limit was set at 10 minutes and best objective valuesand number of network problems solved are reported.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Performance Results
Empirical Testing: Approaches
Three approaches were tested:Optimization: GLPK.js with default MIP techniquesSerial BPSO-FCNFDistributed BPSO-FCNF
Devices used in the DAAP cluster include: MacBookPro,Surface Pro, iPhone, Windows laptop, iPad, Nexus 7tablet
A time limit was set at 10 minutes and best objective valuesand number of network problems solved are reported.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Performance Results
Empirical Testing: Approaches
Three approaches were tested:Optimization: GLPK.js with default MIP techniquesSerial BPSO-FCNFDistributed BPSO-FCNF
Devices used in the DAAP cluster include: MacBookPro,Surface Pro, iPhone, Windows laptop, iPad, Nexus 7tablet
A time limit was set at 10 minutes and best objective valuesand number of network problems solved are reported.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Performance Results
Empirical Testing: Approaches
Three approaches were tested:Optimization: GLPK.js with default MIP techniquesSerial BPSO-FCNFDistributed BPSO-FCNF
Devices used in the DAAP cluster include: MacBookPro,Surface Pro, iPhone, Windows laptop, iPad, Nexus 7tablet
A time limit was set at 10 minutes and best objective valuesand number of network problems solved are reported.
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Performance Results
Empirical Testing: Performance
Computational Time: 10 minutes
GLPK Serial Distributed Serial vs.Problem Opt BPSO-FCNF BPSO-FCNF Distributed
5n6p1.3M 937K 907K (37K)285K 300K 1.3M (4.3x)
10n21pN/A 7.36M 7.21M (150K)5.2K 30K 68K (2.3x)
20n30pN/A 21.16M 20.97M (190k)3.2K 1.2K 3.3K (2.8x)
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Demonstration
I hope this works...
pso.gametheorylabs.com1 Navigate to this website2 Register your device as a Processing Node3 Watch for the green “loaded” and “running” indicators
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Summary
Binary PSO (with modifications) is an effective option fordifficult Fixed-Charge Network Flow problems.
Binary PSO for FCNF are naturally independent and thuseasily parallelized.Using the DAAP technology, it is easy to create dynamiccluster computing to address difficult MIP and problemswith such techniques.Questions?
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Summary
Binary PSO (with modifications) is an effective option fordifficult Fixed-Charge Network Flow problems.Binary PSO for FCNF are naturally independent and thuseasily parallelized.
Using the DAAP technology, it is easy to create dynamiccluster computing to address difficult MIP and problemswith such techniques.Questions?
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Summary
Binary PSO (with modifications) is an effective option fordifficult Fixed-Charge Network Flow problems.Binary PSO for FCNF are naturally independent and thuseasily parallelized.Using the DAAP technology, it is easy to create dynamiccluster computing to address difficult MIP and problemswith such techniques.
Questions?
Introduction Problem Approaches Dynamically Distributed BPSO Approach Performance Results and Demonstration Summary
Summary
Binary PSO (with modifications) is an effective option fordifficult Fixed-Charge Network Flow problems.Binary PSO for FCNF are naturally independent and thuseasily parallelized.Using the DAAP technology, it is easy to create dynamiccluster computing to address difficult MIP and problemswith such techniques.Questions?