View
26
Download
0
Category
Preview:
DESCRIPTION
Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem. Saïd HANAFI Said.hanafi@univ-valenciennes.fr. SYM-OP-IS 2009. Motivation. P : Hard Optimization Problem. Lower Bound v (P). Optimal Value v ( P ). Upper Bound v (P). Maximization. Exact - PowerPoint PPT Presentation
Citation preview
Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem
Saïd HANAFI
Said.hanafi@univ-valenciennes.fr
SYM-OP-IS 2009
2
Motivation
LowerBound
v(P)
UpperBoundv(P)
Heuristic &Metaheuristic
ExactMethod
OptimalValuev(P)
Maximization
Small sizeLarge size Large size
Relaxation &Duality
P : Hard Optimization Problem
3
Outline
Variants of Multidimensional Knapsack problem (MKP)
Tabu Search & Dynamic Programming Relaxations of MKP Iterative Relaxation-based Heuristics Inequalities and Target Objectives
Conclusions
4
Multidimensional Knapsack Problem
MKP is NP-Hard Dantzig 1957, Lorie & Savage 1955, Markowitz & Manne 1957 Martello & Toth, 1990; Kellerer & Pferschy & Pisinger 2004 Fréville & Hanafi 2005, Wilbaut & Hanafi & Salhi 2008; Hanafi & Wilbaut 2009 Boussier & Vasquez & Vimont & Hanafi & Michelon 2009 Hanafi & Lazic & Mladenovic & Wilbaut 2009
nx
bAts
c
MKP
1;0
x ..
x max
A(m,n),b(m),c(n) 0
5
Applications Resource allocation problems Production scheduling problems Transportation management problems Adaptive multimedia system with QoS Telecommunication networks problems Wireless switch design Service selection for web services with QoS constraints Polyedral approach, (1,k)-configuration
Sub-problem : Lagrangean / Surrogate relaxation Benchmark
6
Variants of Knapsack Problem
- Unidimensional Knapsack Problem m = 1
- Bi-Dimensional Knapsack Problem m = 2
- Quadratic- Hyperbolic- Min-Max- Multi-Objectives- Bi-Level
- Cardinality- Multiple-choice- Precedence - Demand- Quadratic
ObjectiveObjective ConstraintsConstraints
7
Multidemand Multidimensional Knapsack Problem
Knapsack & Covering Problem
8
,...,gj,...,gi,x
,...,gix
,...,mkbxats
xc
MMKP
iij
g
jij
kij
g
i
g
j
k
g
i
g
jijij
i
i
ij
i
1 ,11 0
11
1..
max
1
1 1
1 1
Multi-Choice Multidimensional Knapsack Problem
• g : Number of groups• gi : Number of items in group i• cij : Profit associated to item j of group i• : Consuming resource k by item j of group i• bk : Capacity of resource k
kija
9
,...,gj,...,gi,x
,...,gix
,...,mkbxats
xc
GUBMKP
iij
g
jij
kij
g
i
g
j
k
g
i
g
jijij
i
i
ij
i
1 ,11 0
11
1..
max
1
1 1
1 1
Multidimensional Knapsack Problem with Generalized Upper Bound constraints
10
Knapsack Constrained Maximum Spanning Tree Problem
KCMST problem arises in practice in situations where the aim is to design a mobile communication network under a strict limit on total costs.
Aa,x
x
bxwts
xc
KCMST
a
Aaaa
Aaaa
1 0
G of treecovering
..
max
Aggarwal et al (1982)
11
Generalized Knapsack Problem
nj,yx
nj ybxya
cxts
xc
GKP
jj
jjjjj
n
jj
n
jjj
,...,11 0;0
,...,1
..
max
1
1
Suzuki (1978)
12
Disjunctively Constrained Knapsack Problem
nj,x
Ii,jxx
bxats
xc
DCKP
j
ji
n
jjj
n
jjj
,...,11 0
)( 1
..
max
1
1
Yamada et al (2002)
13
Max Min Multidimensional Knapsack
Problem
n
k
xbAx
ts
pkxcMinMax
MMMKP
1,0
..
,...,1:
Introduction to minimax, Demyanov Molozemov 1974
Minimax and applications, Du, Pardalos 1995
Robust optimization applications, Yu 1996
A virtual pegging approach, Taniguchi, Yamada, Kataoka 2009
Iterative Heuristics, Mansi, Hanafi, Wilbaut 2009
14
Nj,x
liNNNN
bxats
mixc
KSP
j
li
m
ii
Njjj
Njjj
i
1 0
,
..
,...,1:minmax
1
Knapsack Sharing ProblemBrown (1979)Brown (1979)
15
Moore, Bard 1990 => B2 = 0Dempe, Richter 2000Hanafi, Mansi, Brotcorne 2009
15
yxMax
, f 1 ( x , y ) = d 1 x + d 2 y
s.t. B1 x + B2 y ≤ b1 x 1nZ
y
Max f 2 ( y ) = c y
s.t. A1 x + A2 y ≤ b2
y 2n1,0
Leader objective function
Follower objective function
Follower problem
(BKP)
Bilevel Knapsack Problem
16
Tabu Search & Dynamic Programming
Metaheuristic: Tabu Search (TS) Very powerful method on many optimization
problems Based on the exploration of a neighbourhood
Exact method: Dynamic Programming (DP) Efficient for small-sized problem Can be used for solving a series of "similar" problem Can be enhanced by adding reduction rules
17
Tabu Search & Dynamic Programming
|N 1| small and depends on the instance and the machine. How defining N 1 and N 2 ?
N1
N2
Dynamic Programmingwith Fixation on N1
Tabu Search on N2
Partition
Recurrence
|N| = 8
18
Dynamic Programming (Bellman, 1957)
Recurrence relation for the MKP
In our case all the subproblems concern the same variables and only g varies
19
Reduction Rules
Let x0 be a feasible solution of the MKP. Order the variables such that:
Proposition 1 (Optimality):
Let ,
Proposition 2 (Fixation):
Balev, Fréville, Yanev and Andonov, 2001.
20
Dynamic Programming as a Heuristic
P Initial solution: x0
Order the variables:uj upper bound of P(e-x0,{j})
Partition the set of variables: N 1 and N 2
Dynamic Programming
if cx1 > cx0
or if |F| > 0
N 1
N 2
n, m
the machine
List L (with all the optimal values)Fixation Fx1
21
Global Intensification (GI) Mechanism
Dynamic Programming
List L (with all the optimal values)Fixation Fx1
Tabu Search:performed on N 2 and uses L
x*
If cx* > cx1 then restart the algorithm with x* as the initial solution
22
? 0 1 ? ? 1 011 0 1 0 1 1 00
Tabu Search
TS on N2 (|N2| = n2) Neighbourhood of the current solution x2 on N2:
Evaluation of a solution over N from a solution over N2
N 1
N 2
Scan in L
23
Computational Environment
Algorithm coded in C language The results have been obtained on a Solaris Ultra
Sparc workstation 300MHz n1 15% of n OR-Library : 270 correlated instances
n = 100, 250, 500 and m = 5, 10, 30 30 instances for each (n,m)
24
A Summary of the Results
Each value = average over 30 instances
%: average gap wrt CPLEX (with the same computational time)
CPU: in seconds
TS alone: Tabu Search algorithm without global intensification mechanism
25
RelaxationDefinition : Let
(P) max{ f(x) : x X}
(R) max{ g(x) : x Y}
Problem R is a relaxation of P if
1) X Y
2) x X : f(x) ≤ g(x).Properties
- v(P) ≤ v(R)
- Let x* an optimal solution of R, if x* is feasible for P and f(x*)=g(x*) then x* is an optimal solution of P
26
LP-RelaxationDantzig (1957)
max max cxcx
(LP)(LP) s.t.s.t. Ax ≤ bAx ≤ b
x x [0, 1] [0, 1]nn
Properties :
- v(MKP) ≤ v(LP)
- Let x* an optimal solution of LP,
if x* {0,1}n then x* is an optimal solution of MKP
27
MIP Relaxation
Let x0 {0, 1}n and J N
Remarks: MIP(x0,) = LP(P), MIP(x0,N) = P v(MIP(x0,J)) v(MIP(x0,J’) if J J’
Stronger bound : v(P) v(MIP(P,J)) v(LP(P))
Hanafi & Wilbaut (06), Glover (06)
28
Lagrangean Relaxation
Lagrangean Relaxation : 0
LR() max{cx + (b – Ax) : x X} Lagrangean Dual :
(L) min{v(LR()) : 0}
Properties :
- v(MKP) ≤ v(L) ≤ v(LR())
- Let x* be an optimal solution of LR(*), if *(b – Ax*) = 0 and x* is feasible for P then x* is an optimal solution of P
Held & Krap (71), Geoffrion (74)
29
Geometric Interpretation : v(L) = max{cx : Ax ≤ b, x co(X)}
eDxx
Conv nZxeDxx ,
bAxx
cx
LP
LR
P
nZxeDxxX ,
30
Lagrangean Decomposition
nZx
eDx
bAx
cxZ
max
n
n
Zy
Zx
yx
eDy
bAx
cxZ
max
n
n
LD
Zy
Zx
eDy
bAx
xyvcxvZ
)(max)(
Guignard & Kim (87)
31
Lagrangean Decomposition
)()(min 210
vZvZZ LDLDv
LD
n
n
LD
Zy
Zx
eDy
bAx
xyvcxvZ
)(max)(
n
LD
Zx
bAx
xvcvZ
)(max)(1
n
LD
Zy
eDy
vyvZ
max)(2
Sub-problem 1 Sub-problem 2
32
P
Geometric InterpretationSub Pb 1
eDxx bAxx
cx
LP
LR
Conv nZxeDxx ,
Conv nZxbxAx ,
LD
Sub Pb 2
33
Theorem
)()()()( LPvLRvLDvPv
34
Surrogate Relaxation
Relaxation surrogate : 0SR()max{cx : Ax ≤ b, x X}
Dual surrogate corresponding : (S) min{v(SR()) : 0}.
Glover (65)
Propriétés :
- v(MKP) ≤ v(S) ≤ v(SR())
- Let x* be an optimal solution of SR(),
if x* is feasible for MKP then x* is an optimal solution of MKP
- (S) min{v(SR()) : U = { 0 : |||| = 1}}
35
Composite Relaxation Composite Relaxation : , ≥ 0
CR(, ) max{cx + (b – Ax) : Ax ≤ b and x X}
Composite Dual :
(C) min{v(CR(, )) : , ≥ 0}
Greenberg & Pierskalla (70)
Remarks : - v(LR()) = v(CR(, 0)) - v(SR()) = v(CR(0, ))
36
Comparisons
v(P) v(C) v(S) v(L) v(LP)
37
Properties of Relaxation Functions
Lagrangean function v(LR()) is convex and piecewise-linear if X is finite
Surrogate function surrogate v(SR()) is quasi-convex and piecewise-linear if X is finite
Composite function v(CR(, )) is non convex, non quasi-convex
38
Methods for Dual
(Quasi-)Sub-gradient : ’ = + t gg : sub-gradient of f( )=v(R())
t : Stepwise
x*() an optimal solution of R() g() = b – Ax*() a sub-gradient of f()
Bundle method, linearization, etc.
39
Intersection Algorithmvv(LR((LR())))
11 2200 maxmax**
Fisher (75), Yu (90), (90), Hanafi (98)
40
Extended Dichotomy Algorithmfor Bi-dimensional Knapsack
AA11xx≤≤bb11
AA22xx≤≤bb22
x*(u)x*(u)
uAxuAx≤u≤ubb
Fréville & Plateau (93), Fréville & Plateau (93), Hanafi (93)Hanafi (93)
41
Relaxation-based Heuristics
u = u°, UB = g(u), LB = f(x0) Repeat
let x*(u) an optimal of the current relaxation R(u) let x(u) the projection of x*(u) an the feasible set of P UB = min(UB, v(R(u))) LB = max(LB, f(x(u))) Update the current multiplier u
42
Relaxation-based Heuristics
Step 1: Solve one or more relaxations of the current problem P to generate one or more constraints.
Step 2: Solve one or more reduced problems induced by the optimal solution(s) of the previous relaxation(s) to obtain one or more feasible solution(s) of the initial problem.
Step 3: If a stopping criteria is satisfied then return the best lower bound. Else add a generated constraint(s) to the problem P and return to step 1.
Hanafi & Wilbaut (2006)
43
Space Search
Binary Solution : x {0, 1}n
Improved Heuristic Continuous Solution : x [0, 1]n
LP-Relaxation Partial Solution : x {0, 1, *}n where * = free
Branch&Bound, Dynamic Programming, Heuristic Feasible and Infeasible Solutions
Oscillation, Resolution Search, Dynamic B&B
44
Reduced Problem
Let x0 {0, 1}n and J N
P(x0, J) = (P| xJ = x0J)
Remarks : P(x0,) = P, v(P(x0,J)) v(P(x0,J’) if J J’ xJ = x0
J (eJ – 2 x0J)xJ + eJ x0
J = 0
|J| = 1 If v(P(e - x0,{j})) ≤ cx0 then x*j = x0
j , x* an optimal solution
45
LP-based Heuristic
Step 1: Solve the LP-relaxation of P to obtain an optimal solution xLP
Step 2: Solve the reduced problem P(xLP,J(xLP)) to generate an optimal solution x0
where J (x) = {j N : xj {0, 1}}
Remarks: cx0 ≤ v(P) ≤ cxLP
Relaxation induced neighbourhoods search, Danna et al., 2003P((xLP+x*)/2,J((xLP+x*))/2) with cx > cx*
where x* the best feasible solution
46
Pseudo-Cut
Let y be a vector in {0, 1}n, the following inequality (*) cuts off the solution y without cutting off any other solution in {0, 1}n
(*)
where J0(x) = {j N : xj = 0} J1(x) = {j N : xj = 1}.
Example:
(*) is called canonical cut in Balas and Jeroslow, 1972.
47
Partial Pseudo cuts
Let x’ be a partial solution in {0, 1, *}n, the inequality
dJ(x, x’) = (eJ – 2 x’J)xJ + eJ x’J 1 (*)cuts off x’ and all partial solution dominated by x’.
J = {j N: x’j {0,1}}, xJ = (xj : j J)Remarks : (*) is called
Pseudo Cut by Glover, 2005. Canonical Cut by Balas & Jeroslow, 1972, if J = N
dN (x, x’) = Hamming distance over {0, 1}n
48
Iterative LP-based heuristic (ILPH)
Let x* a feasible solution of Pv* = cx*; Q = P ; stop = False;while (stop = False) do Let xLP be an optimal solution of LP(Q); J = J(xLP);
Let x0 be an optimal solution of (Q|(eJ – 2 xLPJ)xJ k – eJ xLP
J) if cx0 > v* then x* = x0; v* = cx0
Q = (Q | (eJ – 2 xLPJ)xJ k + 1 – eJ xLP
J) if cxLP-v* < 1 then stop = True end while
49
Theorem : Finite Convergence
The algorithm ILPH converges to an optimal solution of the problem in a finite number of iterations ( 3n).
Proof : v(P) = max(v(P|dx d0), v(P|dx d0 + 1))
d = (eJ – 2 xLPJ, 0) and d0 = – eJ xLP
J
Number of partial solutions = |{0, 1, *}n|=3n
50
Convergence of IPLH
CPU 400sec (Cplex 10sec)
Replace the optimal condition by a total number of iterations
F(x): set of fractional variables of solution x
51
Remark and Dominance
1 001 1 0 *01 * 01
1 001 1 0 *01 * * *
1 001 1 0 *01 * 01
1 001 1 0 *01 * * *x1
x2
J1 J0 F
Reduced problem included
52
Improvements: Dominance Properties
Definition: Let x1 and x2 be two solutions in [0,1]n,x1 dominates x2 if F(x2) F(x1), J 1(x1) J 1(x2) and J 0(x1) J 0(x2)
Property 1: If x1 dominates x2 then v(P(x1,J(x1))) v(P(x2,J(x2)))
Property 2: The solution x defined bydominates x1 and x2, and we have :
v(P(x,J(x))) max { v(P(x1,J(x1))) , v(P(x2,J(x2))) }
Reduction of the number of reduced problems to solve
53
Illustration of Dominance
ILPH_D: ILPH with property 1
ILPH_D*(t) : ILPH with properties 1 and 2
Reduction between 30 and 50%
54
Iterative independent relaxations-based heuristic (IIRH)
xk optimal solution of MIP(Qk,F(xk-1))
k = k + 1
Pk+1 = (Pk | {f kx ≤ | J1(yk)| - 1}
yk optimal solution of LP(Pk)
wk optimal solution of P(yk,J(yk)) zk optimal solution of Q(xk,J(xk))
v*= max(v*,cwk,czk) u = min(cyk,cxk)
Q k+1 = (Q k | {f kx ≤ | J1(xk)| - 1}
Q = P, x0 optimal solution of LP(P); k = 1
Stop No Yes
If u–v*< 1
55
Iterative Relaxations-based Heuristic (IRH)
k = k + 1
yk optimal solution of LP(Pk)
Pk+1 = (Pk | {f kx ≤ | J1(yk)| - 1 and/or f kx ≤ | J1(xk)| - 1}
xk optimal solution of MIP(Pk,F(yk))
wk optimal solution of P(yk,J(yk)) zk optimal solution of P(xk,J(xk))
v*= max(v*,cwk,czk) u = min(cxk,cyk)Stop
No
Yes
If u–v*< 1
56
Comparison on a small instance
n = 30; m = 10
v* = 376
57
Results for n = 500
1 line = average / 10 instances
Average computational time 70 hours 2 hours
58
Summary (n = 500)
1 line = results for 30 instances
59
Some Related Works
Canonical Cuts on the Unit Hypercube, Balas & Jeroslow, 1972 0-1 IP with many variables and few constraints, Soyster, Lev,
Slivka, 1978 A hybrid approach to discrete mathematical programming,
Marsten, Morin, 1978 A new hybrid method combining exact solution and local
search, Mautor, Michelon, 1997 Parametric Branch and Bound, Glover, 1978 Large neighborhood search, Shaw, 1998 Local branching, Fischetti, Lodi, 2002
60
Some Related Works Short Hot Starts for B&B, Spielberg & Guignard, 2003 Relaxation induced neighbourhood search, Danna, Rothberg, Pape, 2003 Adaptive memory projection method, Glover, 2005 Feasibility Pump, Fischetti, Glover, Lodi, 2005 Relaxation guided VNS, Puchinger, Raidl, 2005 Global intensification using dynamic programming, Wilbaut, Hanafi,
Fréville, Balev, 2006 Convergent Heuristics for 0-1 MIP, Wilbaut, Hanafi, 2006 Variable Neighbourhood Decomposition Search, Lazic, Hanafi,
Mladenovic, Urosevic, 2009 Inequalities and Target Objectives in Metaheuristics for MIP, Glover,
Hanafi, 2009
61
Inequalities and Target Objectives in Metaheuristics for MIP
Recent adaptive memory and evolutionary metaheuristics for MIP have included proposals for introducing inequalities and target objectives to guide the search.
Two types of search strategies Fix subsets of variables to particular values within
approaches for exploiting strongly determined and consistent variables,
Use of solution targeting procedures.
Glover & Hanafi (09)
62
Inequalities and Target Objectives
Let a Target Solution x′, solveLP(x′, c′, M) min{Mfx + c′x : x X}
M is a large positive number. The coefficients c′j of the Target Objective seek to induce assignments xj = x′j for different variables with varying degrees of emphasis.An alternative to solve LP(LP(x′, c′, M) in two stages: 1- Solve LP(x′, c′, 0) = min{c′x : x X}2- Solve the problem min{fx : x X, cx′ = v(LP(x′, c′, 0))}Remark : The residual problem can be significantly smaller than the first stage problem, allowing it to be solved very efficiently
Glover & Hanafi (09)
63
Strong Pseudo Cut
x {0, 1}n x X {x : (x, x) max{1, v(LP(x))}} = X
LP(x) min{(x, x) : x X}
We call x the target solution for LP(x’) Remarks : x X v(LP(x)) = 0 (x, x ) = v(LP(x)) fractional x X Parametric branch and bound, Glover (1978) Feasibility pump, Fischetti, Glover and Lodi (2005)
Nj
jjjj xxxxxx )1(')'1(),'(
64
Illustration
MartinGrötsche
l
64
x
x
(x, x) = min{(x, x) : x X}d(x, x) d(x, x)
65
Partial Pseudo Cuts
For all y s.t. (J, x, y) = 0, we have
y X {x : (J, x, x) max{1, v(LP(J, x))}}
LP(J, x) min{(J, x, x) : x X}
Remarks : Partial Hamming distance over {0, 1, *}n
Jj
jjjj xxxxxxJ )1(')'1(),',(
Let x [0,1]n, J N(x) = {j : xj {0, 1}}
66
Theorem
Let x [0,1]n, N(x) = {j N: xj {0, 1}}, k an integer 0 k n - |N(x)|
The canonical hyperplane associated to x :H(x, k) = { x [0,1]n : (N(x), x, x) = k}We have :
Co(H(x, k) {0,1}n) = H(x, k)
where Co(X) is the convex hull of the set X.
Balas & Jeroslow 1972, Glover & Hanafi 2008
67
Weighted Partial Pseudo Cuts
Nj
jjjjj xxxxcxxc )1(')'1(),',(Let x [0,1]n, c C(x) = {c INn : cjxj (1 – xj) = 0}
x {0, 1}n y : (c, x, y) = 0 y X {x : (c, x, x) max{1, v(LP(c, x))}}
LP(c, x) min{(c, x, x) : x X}Remarks : (e, x, x) = || x – x||1 = || x – x||2 (J, x, x) = (c, x, x) : cj = 1 if j J else cj = 0.
68
Additional and Stronger Inequalities from Basic LP Solutions
Let x″ a basic feasible solution for
LP(c, x) min{(c, x, x) : x X}
Let d = c - rc where rc is reduced cost to x″, consider the pseudo cut
(d, x, x) v(LP(d, x)) (*)
LP(d, x) min{(d, x, x) : x X}
The inequality (*) is satisfied by all binary vectors x X, and excludes the solution x = x″ when (c, x,x″) is fractional.
If the basic solution x″ for LP(x, c) is optimal then the inequality (*) dominates
(c, x, x) v(LP(c, x))
69
Intensification Procedure
Pp
PpsXx
xxdsxxdts
sws
p
pppp
ppPp
pp
,
,0,
)",',(),',(..
min 0
Pp
Xx
xxdxxdsts
sppppp
,),',()",',(..
min
0
0
Min-Sum
Min-Max
70
Diversification Procedure
Pp
PpsXx
xxdsxxdts
sws
p
pppp
ppPp
pp
,
,0,
)",',(),',(..
max 0
Pp
Xx
xxdxxdsts
sppppp
,),',()",',(..
max
0
0
Max-Sum
Max-Min
71
Conclusions
Many MIP problems resist to be solved by the best current B&B and B&C methods.
Metaheuristics can likewise profit from generating inequalities to supplement their basic functions.
This framework can be exploited in generating target objectives, employing both adaptive memory ideas and newer ones proposed here.
72
Some Prospects
Extension to other problems Improve the implementation of dominance rules Integrate intensification and diversification Strengthen the use of memory Parallel versions
Recommended