Knapsack Problems with Side Constraints SCATAMACCHIA... · (Yang) and a dynamic programming approach (Chebil) yi= 1 yi= 0 - Branching on families variables leads to solve standard

Knapsack Problemswith Side Constraints

Rosario Scatamacchia

PhD Dissertation Defense (cycle XXIX)

Supervisor: Prof. Federico Della Croce di Dojola

Dipartimento di Automatica e InformaticaPolitecnico di Torino

March 24, 2017Rosario Scatamacchia Knapsack Problems with Side Constraints March 24, 2017 1 / 43

Summary of the PhD activities

Research Topic: methods and applications in CombinatorialOptimization

Academic research:- Knapsack Problems with Side Constraints

- Critical Node Problems (research line with Universita diTorino)

Industrial projects: Energy sector and Internet of Thingsapplications (at TIM Joint Open Lab SWARM)

Teaching : “Ricerca Operativa” course, exercise sessions (2015,2016)

Part of the academic activities within a collaboration with Prof.Ulrich Pferschy (Karl-Franzens University, Graz)

Rosario Scatamacchia Knapsack Problems with Side Constraints March 24, 2017 2 / 43









































..and main results

Algorithms and theoretical contributions for problems inCombinatorial Optimization

Solutions for energy sector and IoT applications

Thesis–related publications:- 3 published papers on international journals

- 2 papers submitted (1 journal, 1 conference)

Publications from other research lines:

- 3 published papers (2 journals, 1 conference)- 1 paper submitted to a journal


..and main results








..and main results








..and main results








Thesis overview

Knapsack Problems with Side Constraints


Thesis overview

Topic choice:

– Challenging variants of the classical KP (NP–hard problems)

– Real-life applications

Main research lines:

Exact solution approaches (byMathematical Programming andDynamic Programming)

Approximation algorithms andtheoretical contributions


Thesis overview

Topic choice:







Thesis overview

Topic choice:







A quick introduction to Knapsack Problems




The 0-1 Knapsack Problem

A set of items j with given profits pj and weights wj is available.The aim is to select a subset of the items so as to maximizethe total profit without exceeding a known knapsack capacity.

Integer Linear Programming (ILP) Formulation

maximizen∑

j=1

pjxj (1)

subject ton∑

j=1

wjxj ≤ c (2)

xj ∈ {0, 1} j = 1, . . . , n (3)

... + many relevantvariants



The 0-1 Knapsack Problem

A set of items j with given profits pj and weights wj is available.The aim is to select a subset of the items so as to maximizethe total profit without exceeding a known knapsack capacity.

Integer Linear Programming (ILP) Formulation

maximizen∑

j=1

pjxj (1)

subject ton∑

j=1

wjxj ≤ c (2)

xj ∈ {0, 1} j = 1, . . . , n (3)

... + many relevantvariants



Applications in resource allocation problems,graph partitioning,scheduling, vehicle routing, cryptography

Strongly investigated problems

Beautiful research lines:

– Dynamic programming

– Branch and Bound algorithms

– Core concept and relatedalgorithms

– Approximation schemes

– Linear Programming

– Exponential algorithms

– Heuristics



Applications in resource allocation problems,graph partitioning,scheduling, vehicle routing, cryptography

Strongly investigated problems

Beautiful research lines:

– Dynamic programming

– Branch and Bound algorithms

– Core concept and relatedalgorithms

– Approximation schemes

– Linear Programming

– Exponential algorithms

– Heuristics


The 0–1 Knapsack Problem with Setups




Notation and Problem Formulation

We consider the 0–1 Knapsack problem with setups (KPS), a variantof KP where items (xij) belong to specific families (yi):

MaxN∑i=1

ni∑j=1

pijxij −N∑i=1

fiyi

s.t.N∑i=1

ni∑j=1

wijxij +N∑i=1

diyi ≤ b

xij ≤ yi j = 1, . . . , ni , i = 1, . . . ,N

xij ∈ {0, 1} j = 1, . . . , ni , i = 1, . . . ,N

yi ∈ {0, 1} i = 1, . . . ,N




We consider the 0–1 Knapsack problem with setups (KPS), a variantof KP where items (xij) belong to specific families (yi):

MaxN∑i=1

ni∑j=1

pijxij −N∑i=1

fiyi

s.t.N∑i=1

ni∑j=1

wijxij +N∑i=1

diyi ≤ b

xij ≤ yi j = 1, . . . , ni , i = 1, . . . ,N

xij ∈ {0, 1} j = 1, . . . , ni , i = 1, . . . ,N

yi ∈ {0, 1} i = 1, . . . ,N



Applications

- Resource allocation problems in manufacturing, cargo loading,marketing

- Link with a Horizon 2020 projectwithin the smart-home paradigm(FLEXMETER project)



Applications

- Resource allocation problems in manufacturing, cargo loading,marketing

- Link with a Horizon 2020 projectwithin the smart-home paradigm(FLEXMETER project)



Previous works

0–1 KPS addressed by Bulfin and Yang(2006,2009), Chebil et al.(2015):

Related works (special cases and other variants): Chajakis andGuignard (1994); Akinc (2006); Altay (2008)

Results: B&B algorithm producing solutions close to optimality(Yang) and a dynamic programming approach (Chebil)

yi = 0yi = 1- Branching on families variablesleads to solve standard KPs..

- ..but this branching strategyineffective as soon as the numberof families increases



Previous works








Previous works








Previous works








An exact enumerative approach (FLEA)

FLEA algorithm (First Level variables Exploration based Approach)

Main idea:

Effective exploration of the solution space of the families: Identify thepossible optimal number of families and look for solutions withinthis range

Three steps:

Computing a first solution and fixing variables

Identifying the relevant range of sums of families

Exploring the detected solution space by an iterative procedure





Main idea:


Three steps:








Main idea:


Three steps:








Main idea:


Three steps:








Main idea:


Three steps:






FLEA algorithm

First step: Computing a first solution and fixing variables

1 Solve the linear relaxation of the problem, requiring thatN∑i=1

yi

must be integer

2 Give the target sum S , solve the MILP problem with binary yi ,

continuous xij andN∑i=1

yi = S

3 Set yi variables to 0/1 according to 2). Solve the inducedclassical KP

Fast procedure to get a feasible KPS solution (LB ′)

+ Variables fixing by analysis of LP relaxation results (reduced costs).



FLEA algorithm



yi

must be integer



yi = S






FLEA algorithm



yi

must be integer



yi = S






FLEA algorithm



yi

must be integer



yi = S






FLEA algorithm



yi

must be integer



yi = S






FLEA algorithm



yi

must be integer



yi = S






FLEA algorithm

Second step: Identifying the relevant range of families

Interval of possibly optimal sums of families [Smin, Smax ]straightforwardly calculated by solving two continuous problems:

Min (Max)N∑i=1

yi

s.t.N∑i=1

ni∑j=1

wijxij +N∑i=1

diyi ≤ b

xij ≤ yi j = 1, . . . , ni ,N∑i=1

ni∑j=1

pijxij −N∑i=1

fiyi ≥ LB ′ + 1

0 ≤ xij ≤ 1 ∀ j = 1, . . . , ni , ∀ i = 1, . . . ,N

0 ≤ yi ≤ 1 ∀ i = 1, . . . ,N

..What remains to do:

To explore sub–problems wherethe sum of yi ranges in [Smin, Smax ]



FLEA algorithm



Min (Max)N∑i=1

yi

s.t.N∑i=1

ni∑j=1

wijxij +N∑i=1

diyi ≤ b

xij ≤ yi j = 1, . . . , ni ,N∑i=1

ni∑j=1

pijxij −N∑i=1

fiyi ≥ LB ′ + 1

0 ≤ xij ≤ 1 ∀ j = 1, . . . , ni , ∀ i = 1, . . . ,N

0 ≤ yi ≤ 1 ∀ i = 1, . . . ,N





FLEA algorithm



Min (Max)N∑i=1

yi

s.t.N∑i=1

ni∑j=1

wijxij +N∑i=1

diyi ≤ b

xij ≤ yi j = 1, . . . , ni ,N∑i=1

ni∑j=1

pijxij −N∑i=1

fiyi ≥ LB ′ + 1

0 ≤ xij ≤ 1 ∀ j = 1, . . . , ni , ∀ i = 1, . . . ,N

0 ≤ yi ≤ 1 ∀ i = 1, . . . ,N





FLEA algorithm



Min (Max)N∑i=1

yi

s.t.N∑i=1

ni∑j=1

wijxij +N∑i=1

diyi ≤ b

xij ≤ yi j = 1, . . . , ni ,N∑i=1

ni∑j=1

pijxij −N∑i=1

fiyi ≥ LB ′ + 1

0 ≤ xij ≤ 1 ∀ j = 1, . . . , ni , ∀ i = 1, . . . ,N

0 ≤ yi ≤ 1 ∀ i = 1, . . . ,N





FLEA algorithm

Third step: Exploring the detected solution space

Let LB∗ be the current best solution value. For each sub–problem:

1 Solve the sub–problem by removing the integrality constraint onxij . This yield an Upper Bound UB

2 If UB ≥ LB∗ + 1, consider the combination of yi found in 1) andactivate the corresponding familiesElse STOP

3 Find the optimal solution of the induced standard Knapsack withvalue LB (If LB > LB∗, LB∗ = LB)

4 Add the constraint: at least one of the families of the previouscombination must be discarded. Then go to 1)



FLEA algorithm









FLEA algorithm









FLEA algorithm









FLEA algorithm









FLEA algorithm









FLEA algorithm

Example:Families: 10 , Items: 1000

Sub–problem:N∑i=1

yi = 5; LB∗ = 22769;

1 UB1 = 22777.8; y1 + y4 + y5 + y9 + y10 = 5; LB1 = 22769

2 Add the constraint: y1 + y4 + y5 + y9 + y10 ≤ 4

3 UB2 = 22776.7; y1 + y4 + y5 + y6 + y9 = 5; LB2 = 22773LB∗ = LB2 = 22773


5 UB3 = 22756.6 ≤ LB∗, STOP



FLEA algorithm



yi = 5; LB∗ = 22769;

1 UB1 = 22777.8; y1 + y4 + y5 + y9 + y10 = 5; LB1 = 22769


3 UB2 = 22776.7; y1 + y4 + y5 + y6 + y9 = 5; LB2 = 22773LB∗ = LB2 = 22773


5 UB3 = 22756.6 ≤ LB∗, STOP



FLEA algorithm



yi = 5; LB∗ = 22769;

1 UB1 = 22777.8; y1 + y4 + y5 + y9 + y10 = 5; LB1 = 22769


3 UB2 = 22776.7; y1 + y4 + y5 + y6 + y9 = 5; LB2 = 22773LB∗ = LB2 = 22773


5 UB3 = 22756.6 ≤ LB∗, STOP



FLEA algorithm



yi = 5; LB∗ = 22769;

1 UB1 = 22777.8; y1 + y4 + y5 + y9 + y10 = 5; LB1 = 22769


3 UB2 = 22776.7; y1 + y4 + y5 + y6 + y9 = 5; LB2 = 22773LB∗ = LB2 = 22773


5 UB3 = 22756.6 ≤ LB∗, STOP



FLEA algorithm



yi = 5; LB∗ = 22769;

1 UB1 = 22777.8; y1 + y4 + y5 + y9 + y10 = 5; LB1 = 22769


3 UB2 = 22776.7; y1 + y4 + y5 + y6 + y9 = 5; LB2 = 22773LB∗ = LB2 = 22773


5 UB3 = 22756.6 ≤ LB∗, STOP



FLEA algorithm



yi = 5; LB∗ = 22769;

1 UB1 = 22777.8; y1 + y4 + y5 + y9 + y10 = 5; LB1 = 22769


3 UB2 = 22776.7; y1 + y4 + y5 + y6 + y9 = 5; LB2 = 22773LB∗ = LB2 = 22773


5 UB3 = 22756.6 ≤ LB∗, STOP



Dynamic Programming for KPS

We also introduce a profit-based dynamic programming algorithm:

- Alternative to the use of solvers

- Pseudopolynomial time algorithm for deriving approximation results(FPTAS)

Insights:

Connection with DP for theKnapsack Problem withPrecedence Constraints

The dynamic program relies on a proper exploration of thesub–problems in KPS tree

Complexity O(nUB) (in practice much lower)







Insights:










Insights:










Insights:






Computational results

We compared our approaches with the state of art solver CPLEX12.5 and algorithms in the literature

Tests showed that both our methods and CPLEX easily can dealwith instances with no correlation between profits and weights ofthe items

We focused on correlated instances as in Chebil (2015):- strong correlation between profits and weights of the items(pij = wij + k)- correlation between setup costs (resp. setup consumption) ofthe families and the profits (resp. weights) of the associateditems
















CPLEX 12.5 Algorithm FLEA Dynamic Progr. from Chebil et al.(Intel i5, 3.3 GHZ) (Intel i5, 3.3 GHZ) (Intel i3, 2.1 GHZ)

Average Max Average Max Average MaxN n time (s) time (s) #Opt time (s) time (s) #Opt time (s) time (s) #Opt5 500 44.17 218.18 10 0.43 0.72 10 0.31 0.49 10

1000 568.37 1200.00 7 0.51 0.66 10 0.92 1.06 102500 1106.42 1200.00 1 0.98 1.28 10 5.34 5.71 105000 929.04 1200.00 3 1.57 1.84 10 20.81 21.52 10

10000 987.01 1200.00 2 3.03 3.67 10 83.93 85.19 1010 500 71.73 423.98 10 0.46 0.64 10 1.50 11.32 10

1000 1200.00 1200.00 0 0.47 0.67 10 1.27 1.38 102500 1200.00 1200.00 0 0.84 1.01 10 7.33 7.72 105000 825.85 1200.00 4 1.47 1.62 10 29.18 30.52 10

10000 1200.00 1200.00 0 3.10 3.48 10 149.73 154.61 1020 500 382.23 1200.00 7 0.61 1.14 10 0.56 0.78 10

1000 50.76 229.96 10 0.51 0.87 10 2.15 2.63 102500 1200.00 1200.00 0 0.88 1.40 10 13.01 13.68 105000 1054.83 1200.00 2 1.58 1.95 10 53.45 54.99 10

10000 1200.00 1200.00 0 2.96 3.74 10 346.58 353.68 1030 500 237.75 1200.00 9 1.62 4.73 10 0.76 0.89 10

1000 499.63 1200.00 8 0.87 1.95 10 3.32 3.63 102500 1175.79 1200.00 1 0.99 1.25 10 19.58 20.20 105000 380.40 1200.00 8 1.62 2.59 10 79.76 83.42 10

10000 907.74 1200.00 5 4.82 8.07 10 526.61 549.03 10

50 20000 685.84 1200.00 5 8.27 14.81 10 - - -...

100 50000 1116.64 1200.00 1 89.39 497.89 10 - - -...

200 100000 1113.78 1200.00 1 163.02 359.18 10 - - -




CPLEX 12.5 Algorithm FLEA Dynamic Progr. from Chebil et al.(Intel i5, 3.3 GHZ) (Intel i5, 3.3 GHZ) (Intel i3, 2.1 GHZ)

Average Max Average Max Average MaxN n time (s) time (s) #Opt time (s) time (s) #Opt time (s) time (s) #Opt5 500 44.17 218.18 10 0.43 0.72 10 0.31 0.49 10

1000 568.37 1200.00 7 0.51 0.66 10 0.92 1.06 102500 1106.42 1200.00 1 0.98 1.28 10 5.34 5.71 105000 929.04 1200.00 3 1.57 1.84 10 20.81 21.52 10

10000 987.01 1200.00 2 3.03 3.67 10 83.93 85.19 1010 500 71.73 423.98 10 0.46 0.64 10 1.50 11.32 10

1000 1200.00 1200.00 0 0.47 0.67 10 1.27 1.38 102500 1200.00 1200.00 0 0.84 1.01 10 7.33 7.72 105000 825.85 1200.00 4 1.47 1.62 10 29.18 30.52 10

10000 1200.00 1200.00 0 3.10 3.48 10 149.73 154.61 1020 500 382.23 1200.00 7 0.61 1.14 10 0.56 0.78 10

1000 50.76 229.96 10 0.51 0.87 10 2.15 2.63 102500 1200.00 1200.00 0 0.88 1.40 10 13.01 13.68 105000 1054.83 1200.00 2 1.58 1.95 10 53.45 54.99 10

10000 1200.00 1200.00 0 2.96 3.74 10 346.58 353.68 1030 500 237.75 1200.00 9 1.62 4.73 10 0.76 0.89 10

1000 499.63 1200.00 8 0.87 1.95 10 3.32 3.63 102500 1175.79 1200.00 1 0.99 1.25 10 19.58 20.20 105000 380.40 1200.00 8 1.62 2.59 10 79.76 83.42 10

10000 907.74 1200.00 5 4.82 8.07 10 526.61 549.03 1050 20000 685.84 1200.00 5 8.27 14.81 10 - - -...

100 50000 1116.64 1200.00 1 89.39 497.89 10 - - -...

200 100000 1113.78 1200.00 1 163.02 359.18 10 - - -




Algorithm New Dynamic Dynamic Progr.FLEA Programming from Chebil et al.

(Intel i5, 3.3 GHZ) (Intel i5, 3.2 GHZ) (Intel i3, 2.1 GHZ)Average Max Average Max Average Max

N n time (s) time (s) time (s) time (s) time (s) time (s)5 500 0.43 0.72 0.02 0.02 0.31 0.49

1000 0.51 0.66 0.07 0.08 0.92 1.062500 0.98 1.28 0.54 0.56 5.34 5.715000 1.57 1.84 2.00 2.16 20.81 21.52

10000 3.03 3.67 8.67 8.98 83.93 85.1910 500 0.46 0.64 0.01 0.02 1.50 11.32

1000 0.47 0.67 0.07 0.08 1.27 1.382500 0.84 1.01 0.49 0.50 7.33 7.725000 1.47 1.62 1.79 1.84 29.18 30.52

10000 3.10 3.48 7.15 7.33 149.73 154.6120 500 0.61 1.14 0.02 0.02 0.56 0.78

1000 0.51 0.87 0.07 0.07 2.15 2.632500 0.88 1.40 0.42 0.45 13.01 13.685000 1.58 1.95 1.68 1.70 53.45 54.99

10000 2.96 3.74 6.69 6.71 346.58 353.6830 500 1.62 4.73 0.02 0.03 0.76 0.89

1000 0.87 1.95 0.06 0.07 3.32 3.632500 0.99 1.25 0.45 0.46 19.58 20.205000 1.62 2.59 1.66 1.68 79.76 83.42

10000 4.82 8.07 6.58 6.74 526.61 549.03



Approximation results

- The standard Knapsack Problem admits PTAS and FPTAS basicschemes. For KPS we got a negative approximation result:

Theorem

KPS does not have a polynomial time approximation algorithm witha bounded approximation ratio unless P = NP .

Sketch of the proof:

Consider a reduction from Subset Sum Problem (SSP) to KPS

Build a KPS instance with optimal solution value 0 if SSP hasnot a solution, 1 if SSP admits a solution

A polynomial time approximation algorithm yielding a positivesolution value would decide the SSP in polynomial time whichimplies P = NP!





Theorem










Theorem










Theorem










Theorem









Given this negative approximation result, we investigated relevantspecial cases of KPS (n items, N families):

Each family can be packed (di +∑

wij ≤ b):

FPTAS running in O(n3 1ε)

Bounded setup costs (fi ≤ max(pij)):

No approximation algorithms

Bounded setup costs (fi ≤ k min(pij)):

FPTAS running in O(n3 1ε

+ nk).

Families of bounded size (ni ≤ C ):

FPTAS running in O(N2

ε2C ).


The 0–1 Collapsing Knapsack Problem





In the 0–1 Collapsing Knapsack Problem (CKP), the capacity isinversely related to the number of items placed inside the knapsack

Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤ B

(n∑

i=1

xi

)xi ∈ {0, 1} i = 1, . . . , n

- B(·) non-increasing function ofthe number of items

- Applications: problems withstructural overheads (e.g.time-sharing computer systems)

- Instances considered in literaturewith up to 1000 items





Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤ B

(n∑

i=1

xi

)xi ∈ {0, 1} i = 1, . . . , n








Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤ B

(n∑

i=1

xi

)xi ∈ {0, 1} i = 1, . . . , n








Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤ B

(n∑

i=1

xi

)xi ∈ {0, 1} i = 1, . . . , n








Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤ B

(n∑

i=1

xi

)xi ∈ {0, 1} i = 1, . . . , n






Overview of the results

A novel linear mathematical formulation:

Idea: to introduce a new set of 0-1 variables (yj) to deal with B(·)

Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤h∑

j=1

bjyj

h∑j=1

yj = 1

n∑i=1

xi =h∑

j=1

jyj

xi ∈ {0, 1} i = 1, . . . , n

yj ∈ {0, 1} j = 1, . . . , h

Added value for the problem:

- Instances with up to 30000items solved within 600 seconds bya MILP solver (CPLEX 12.5)

- New challenging variants can beconsidered (M-CKP)

- The partitioning of the variablesinto two levels can be exploitedagain!






Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤h∑

j=1

bjyj

h∑j=1

yj = 1

n∑i=1

xi =h∑

j=1

jyj

xi ∈ {0, 1} i = 1, . . . , n

yj ∈ {0, 1} j = 1, . . . , h










Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤h∑

j=1

bjyj

h∑j=1

yj = 1

n∑i=1

xi =h∑

j=1

jyj

xi ∈ {0, 1} i = 1, . . . , n

yj ∈ {0, 1} j = 1, . . . , h










Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤h∑

j=1

bjyj

h∑j=1

yj = 1

n∑i=1

xi =h∑

j=1

jyj

xi ∈ {0, 1} i = 1, . . . , n

yj ∈ {0, 1} j = 1, . . . , h










Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤h∑

j=1

bjyj

h∑j=1

yj = 1

n∑i=1

xi =h∑

j=1

jyj

xi ∈ {0, 1} i = 1, . . . , n

yj ∈ {0, 1} j = 1, . . . , h










Maxn∑

i=1

pixi

s.t.n∑

i=1

wixi ≤h∑

j=1

bjyj

h∑j=1

yj = 1

n∑i=1

xi =h∑

j=1

jyj

xi ∈ {0, 1} i = 1, . . . , n

yj ∈ {0, 1} j = 1, . . . , h








An Exact Approach resembling FLEA algorithm for KPS:

Further exploits the peculiarities of the problem

Improves the computational performance of CPLEX 12.5:

For CKP, instances with up to 100000 items solved to optimalitywithin 600 sec

Better deals with the M-CKP variants, especially for 2-CKP:

100000-items instances solved to optimality within 600 sec

(CPLEX 12.5 limited to 10000-items instances)




















































The 0–1 Penalized Knapsack Problem





PKP: n items and a capacity c are given. Each item j has a profit pj ,a weight wj and a penalty πj .The goal is to maximize the sum of the profits minus the greatestpenalty value of the selected items in the solution.

Maxn∑

j=1

pjxj − Π

s.t.n∑

j=1

wjxj ≤ c

πjxj ≤ Π j = 1, . . . , n

xj ∈ {0, 1} j = 1, . . . , n

Π ∈ R

Applications:

- Resource allocation problemswith a bi–objective functions (e.g.batch production systems)





Maxn∑

j=1

pjxj − Π

s.t.n∑

j=1

wjxj ≤ c

πjxj ≤ Π j = 1, . . . , n

xj ∈ {0, 1} j = 1, . . . , n

Π ∈ R

Applications:






Maxn∑

j=1

pjxj − Π

s.t.n∑

j=1

wjxj ≤ c

πjxj ≤ Π j = 1, . . . , n

xj ∈ {0, 1} j = 1, . . . , n

Π ∈ R

Applications:




Previous works

PKP addressed by Ceselli A. & Righini G. (2006):

- A dynamic programming approach is recalled with time complexityO(n2c)

- An exact enumerative algorithm which aims to identify the optimalleading item (i.e. the item that gives the optimal penalty value)

- Large tests were carried out on different instances with up to 10000variables



Previous works







Previous works







Some insights

Characterization of the linear relaxation of PKP:

Theorem

zLP(Π) is a piecewise-linear concave function in Π consisting of atmost 2n linear segments.

...and an algorithm for its solution running in O(n log n)

A “surprising” negative approximation result:

Theorem

PKP does not have a polynomial time approximation algorithm witha bounded approximation ratio unless P = NP .



Some insights


Theorem




Theorem




Some insights


Theorem




Theorem




Exact solution approaches

We first outline a basic dynamic program with running time inO(max{n log n, nc})

Algorithm DP1

Sort items by increasing penalty and run the classical dynamicprogramming by weights for the associated KP once

- Fj(d): optimal solution value of KP sub–problem with items1, . . . , j and capacity d ≤ c

- Given the sorting of the items, the optimal PKP solution value isgiven by

maxj=0,...,n−1

{Fj(c − wj+1) + pj+1 − πj+1}





Algorithm DP1




maxj=0,...,n−1

{Fj(c − wj+1) + pj+1 − πj+1}





Algorithm DP1




maxj=0,...,n−1

{Fj(c − wj+1) + pj+1 − πj+1}





Algorithm DP1




maxj=0,...,n−1

{Fj(c − wj+1) + pj+1 − πj+1}




We propose a core-based solution approach. Two main steps:

Computing a first solution and the relevant interval of penaltyvalues

Running a core-based dynamic programming algorithm

Rationale of the approach:

- For KP, the most effective algorithms are based on the core problem(Minknap, Combo)

- In PKP, the presence of penalties compromises the structure of anoptimal solution with respect to KP

- Narrowing the interval of relevant penalties may reduce the “noise”added by penalties





















































A core-based solution approach

1) Computing a first solution and the relevant interval ofpenalty values

Sketch of the step:

A variant of Minknap algorithm solves two standard knapsackproblems. Results: a PKP solution, some items possiblydiscarded

We then identify the relevant interval of penalties [Πmin,Πmax ]by comparison with upper bounds for specific sub–problems

Output: a first PKP solution (with value z), a reduced PKPinstance (PKP ′) with n′ ≤ n items and the additional constraintΠ ≥ Πmin





Sketch of the step:








Sketch of the step:








Sketch of the step:







2) A core-based dynamic programming algorithm (DP2)

Revisiting of Minknap searching in PKP ′ for better solutions than z

Preliminaries:

Order the items j = 1, . . . , n′ by decreasingpjwj

Define an expanding core as the interval of itemsCa,b = {a, . . . , b} and the set of 0/1 vectors

Xa,b = {xj ∈ {0, 1}, j ∈ Ca,b}






Preliminaries:



Xa,b = {xj ∈ {0, 1}, j ∈ Ca,b}






Preliminaries:



Xa,b = {xj ∈ {0, 1}, j ∈ Ca,b}



DP2 algorithm

In any iteration of the algorithm we have:

xj = 1︷︸︸︷x1, . . . , xa−1,Ca,b,

xj = 0︷︸︸︷xb+1, . . . , xn′

We associate then each partial vector x ∈ Xa,b with a state(ν, µ, πcore , πtot) where:

1 ν =a−1∑j=1

pj +b∑

j=apj xj ;

2 µ =a−1∑j=1

wj +b∑

j=awj xj ;

3 πcore = maxj=a,...,b

{πj : xj = 1};4 πtot = max{πcore , max

j=1,...,a−1πj}.



DP2 algorithm

In any iteration of the algorithm we have:

xj = 1︷︸︸︷x1, . . . , xa−1,Ca,b,

xj = 0︷︸︸︷xb+1, . . . , xn′

We associate then each partial vector x ∈ Xa,b with a state(ν, µ, πcore , πtot) where:

1 ν =a−1∑j=1

pj +b∑

j=apj xj ;

2 µ =a−1∑j=1

wj +b∑

j=awj xj ;

3 πcore = maxj=a,...,b

{πj : xj = 1};4 πtot = max{πcore , max

j=1,...,a−1πj}.



DP2 algorithm

Dynamic programming with states

Starting from the split solution, DP2 enlarges the set Xa,b byalternately evaluating the removal (resp. insertion) of an item fromthe left (resp. right)

xj = 1←︷︸︸︷x1, . . . , xa−1 Xa,b

→ xj = 0︷︸︸︷xb+1, . . . , xn′

Dominance rule between states:

State j is said to be dominated by state i if (ν profits, µ weights)

µi ≤ µj , ν i ≥ ν j , ρi ≥ ρj

where ρ = ν −max{πcore ,Πmin}



DP2 algorithm


Starting from the split solution, DP2 enlarges the set Xa,b byalternately evaluating the removal (resp. insertion) of an item fromthe left (resp. right)

xj = 1←︷︸︸︷x1, . . . , xa−1 Xa,b

→ xj = 0︷︸︸︷xb+1, . . . , xn′

Dominance rule between states:

State j is said to be dominated by state i if (ν profits, µ weights)

µi ≤ µj , ν i ≥ ν j , ρi ≥ ρj

where ρ = ν −max{πcore ,Πmin}Rosario Scatamacchia Knapsack Problems with Side Constraints March 24, 2017 37 / 43


DP2 algorithm


Each iteration has a linear running time with the number ofstates which is bounded by O(n′c) with (n′ ≤ n)

DP2 complexity: O(min{2n, n2c})

This time bound is very pessimistic. In practice...



DP2 algorithm







DP2 algorithm







DP2 algorithm

...we may get a considerable speed up of the performance by:

An enforced dominance rule

Reduction of states through (constant time) upper bound test

Upper bound tests for limiting the insertion of variables in thecore: upper bounds in constant time + enumerative bounds



DP2 algorithm







DP2 algorithm







DP2 algorithm








We consider the generation scheme of the instances in the literature(Ceselli and Righini (2006)):

- Weights distributed in [1,R] or equal to R2

+ v with v in [0, R2

](R = 1000, 10000)

- Different correlation of penalties/profits with weights:

π type Correlation p typeπ1 No correlation p1π2 Weak correlation p2π3 Strong correlation p3π4 Inverse strong correlation p4π5 Almost strong correlation p5π6 Subset-sum correlation p6π7 Constant perimeterπ8 Constant area

Profit = area p7







](R = 1000, 10000)



Profit = area p7







](R = 1000, 10000)



Profit = area p7




Summary results for instances with 1000 items: time (s) and numberof optima over 120 instances (Intel i7 CPU @ 2.4 GHz)

n = 1000 CPLEX 12.5 Ceselli & Righini Algorithm Exact approachProfit Weight Average Max Average Max Average Maxclass type time (s) time (s) #Opt time (s) time (s) #Opt time (s) time (s) #Optp1 a1 0.18 0.25 120 0.00 0.01 120 0.00 0.00 120

a2 0.19 0.48 120 0.00 0.01 120 0.00 0.03 120p2 a1 0.39 1.69 120 0.00 0.01 120 0.00 0.10 120

a2 1.12 6.32 120 0.00 0.01 120 0.01 0.27 120p3 a1 3.90 100.00 117 0.04 0.89 120 0.01 0.40 120

a2 6.83 100.00 117 0.50 8.02 120 0.02 0.29 120p4 a1 59.56 100.00 57 0.10 1.38 120 0.02 0.18 120

a2 66.97 100.00 46 0.28 7.75 120 0.04 0.26 120p5 a1 4.13 100.00 117 0.03 0.89 120 0.02 0.20 120

a2 14.97 100.00 114 0.41 4.50 120 0.07 1.40 120p6 a1 2.67 90.38 120 0.00 0.06 120 0.00 0.03 120

a2 2.97 13.57 120 0.01 0.15 120 0.01 0.08 120p7 a1 27.58 100.00 89 0.00 0.01 120 0.00 0.02 120

a2 38.24 100.00 75 0.01 0.06 120 0.01 0.07 120




Summary results for instances with 10000 items: time (s) andnumber of optima over 120 instances (Intel i7 CPU @ 2.4 GHz)

n = 10000 CPLEX 12.5 Ceselli & Righini Algorithm Exact approachProfit Weight Average Max Average Max Average Maxclass type time (s) time (s) #Opt time (s) time (s) #Opt time (s) time (s) #Optp1 a1 0.95 2.68 120 0.01 0.02 120 0.01 0.03 120

a2 4.19 100.00 117 0.01 0.04 120 0.01 0.03 120p2 a1 5.41 33.84 120 0.01 0.02 120 0.02 0.13 120

a2 12.43 100.00 114 0.01 0.07 120 0.04 0.73 120p3 a1 44.61 100.00 74 25.59 100.00 96 2.59 58.46 120

a2 74.13 100.00 46 48.26 100.00 74 5.53 29.00 120p4 a1 91.60 100.00 11 17.68 100.00 106 2.81 18.88 120

a2 94.69 100.00 7 26.34 100.00 106 7.82 80.02 120p5 a1 25.45 100.00 95 10.70 100.00 113 2.66 70.71 120

a2 65.99 100.00 48 23.74 100.00 101 7.04 58.60 120p6 a1 83.08 100.00 58 0.67 40.17 120 0.22 5.38 120

a2 81.05 100.00 44 3.62 100.00 119 1.95 16.65 120p7 a1 51.00 100.00 63 0.17 0.94 120 0.42 2.50 120

a2 40.12 100.00 75 0.54 3.94 120 1.41 11.46 120


Thesis-related publications

Della Croce, F., Salassa, F., Scatamacchia, R.: A new exact approach for the 0-1Collapsing Knapsack Problem. European Journal of Operational Research. 260:56-69,2017.

Della Croce, F., Salassa, F., Scatamacchia, R.: An exact approach for the 0-1 knapsackproblem with setups. Computers & Operations Research, 80:61-67, 2017.

Pferschy, U., Scatamacchia, R.: Improved dynamic programming and approximationresults for the knapsack problem with setups. International Transactions in OperationalResearch. (on line) 2017.

Della Croce, F., Pferschy, U., Scatamacchia, R.: New exact approaches and approximationresults for the Penalized Knapsack Problem. Submitted to: Discrete Applied Mathematics.

Della Croce, F., Pferschy, U., Scatamacchia, R.: Approximation results for the IncrementalKnapsack Problem. Submitted to: IWOCA 2017 conference.

THANK YOU!


Thesis-related publications

Della Croce, F., Salassa, F., Scatamacchia, R.: A new exact approach for the 0-1Collapsing Knapsack Problem. European Journal of Operational Research. 260:56-69,2017.

Della Croce, F., Salassa, F., Scatamacchia, R.: An exact approach for the 0-1 knapsackproblem with setups. Computers & Operations Research, 80:61-67, 2017.

Pferschy, U., Scatamacchia, R.: Improved dynamic programming and approximationresults for the knapsack problem with setups. International Transactions in OperationalResearch. (on line) 2017.

Della Croce, F., Pferschy, U., Scatamacchia, R.: New exact approaches and approximationresults for the Penalized Knapsack Problem. Submitted to: Discrete Applied Mathematics.

Della Croce, F., Pferschy, U., Scatamacchia, R.: Approximation results for the IncrementalKnapsack Problem. Submitted to: IWOCA 2017 conference.

THANK YOU!


Documents

Knapsack Problems with Side Constraints SCATAMACCHIA... · (Yang) and a dynamic programming approach (Chebil) yi= 1 yi= 0 - Branching on families variables leads to solve standard