Can semidefinite programming be a key approach to the ... · Semide nite programming (or SDP) is an extension of linear programming over the cone of positive semide nite matrices

University of Edinburgh

School of Mathematics

Year 4 Project

Can semidefinite programmingbe a key approach to the pooling

problem?

Author:Elena Zanini

Supervisor:Dr. Andreas Grothey

22nd March 2013

Can semidefinite programming be a key approach to the pooling problem? i

Acknowledgements

Many thanks to my supervisor Andreas Grothey for his extensive patience andguidance throughout the research process, and for the constructive comments andannotations in the writing of the report.I would also like to thank Prof. Jacek Gondzio for his helpful input and valuableadvice.

i

Abstract

Semidefinite programming (or SDP) is an extension of linear programming overthe cone of positive semidefinite matrices. It has become a popular field sincethe 1990s, thanks to its application to combinatorial optimization and controltheory problems, and the development of efficient interior-point algorithms fortheir solution. Here we introduce the main theoretical notions, and proceed toapply them to the pooling problem, a common problem in the petroleum indus-try. This models the production of blends according to some given requirements,using intermediate mixes of the original raw materials.This paper proposes a semidefinite programming relaxation of the pooling prob-lem, modelled originally as a nonconvex and nonlinear program. An algorithm isformulated to implement the relaxation, where the bilinear terms in the modelare expressed as positive semidefinite matrices. This approach causes an excessiverelaxation of the feasible region, leading to poor bounds. Hence, some redundantconstraints, valid inequalities, are deduced and added to the model, in order totighten it and improve the results. An account of the computational work isproduced, together with the results of the numerical tests. The values obtainedare compared to some known optima for the problems, often showing a consid-erable gap. Some suggestions for further work are presented, specifically on thecomputational implementation, the testing process, and the mathematical model.

This project report is submitted in partial fulfilment of the requirements for thedegree of Applied Mathematics BSc.

ii

Contents

Acknowledgements i

Abstract ii

Contents iv

Introduction 1

1 Semidefinite Programming (SDP) 31.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1.1 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2 Interior-point methods . . . . . . . . . . . . . . . . . . . . . . . . 61.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.3.1 Combinatorial optimisation . . . . . . . . . . . . . . . . . 10

2 The Pooling Problem 142.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.2 Parametrisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.3 Different formulations . . . . . . . . . . . . . . . . . . . . . . . . . 182.4 Geometric interpretation . . . . . . . . . . . . . . . . . . . . . . . 192.5 What’s so difficult? . . . . . . . . . . . . . . . . . . . . . . . . . . 202.6 Approaches so far . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3 A Semidefinite Programming relaxation for the Pooling Problem 233.1 Theoretical contributions and proposed developments . . . . . . . 233.2 Model formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 253.3 The proposed algorithm . . . . . . . . . . . . . . . . . . . . . . . 29

4 Examples, results and further work 314.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314.2 Tests and progress . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.2.1 Program formulation . . . . . . . . . . . . . . . . . . . . . 334.2.2 First round of tests . . . . . . . . . . . . . . . . . . . . . . 334.2.3 Second round of tests . . . . . . . . . . . . . . . . . . . . . 344.2.4 Third round of tests . . . . . . . . . . . . . . . . . . . . . 384.2.5 Results and further ideas . . . . . . . . . . . . . . . . . . . 38

4.3 Is there scope for improvement? . . . . . . . . . . . . . . . . . . . 414.3.1 Writing a versatile code . . . . . . . . . . . . . . . . . . . 41

iii

Can semidefinite programming be a key approach to the pooling problem? iv

4.3.2 Running additional tests and analysis . . . . . . . . . . . . 424.3.3 Considering more valid inequalities . . . . . . . . . . . . . 43

4.4 Valid inequalities: summary . . . . . . . . . . . . . . . . . . . . . 45

Conclusions 46

Bibliography 47

iv

Introduction

Many real-life situations can be modelled from an optimisation perspective, andcan be formulated mathematically to produce an optimisation problem of certaintype (which depends on specific characteristics of the model). Historically, opti-misation began to take form in the 1940’s from the use of mathematical techniquesfor tackling specific aspects of military operations (hence the term operations oroperational research)[20]. Examples of applications currently cover a broad rangeof fields, from airlines scheduling to blending raw materials in oil refineries, frombiological to economics issues. In general, these involve trying to answer questionssuch as: what is the best route for the postman to deliver mail? How should weinvest money in the stock market? How detailed an inventory should one keep?What is the best way to deliver radiation therapy?

Often, these questions can be expressed mathematically as picking, from a setof feasible options, the element (or combination of elements) that best satisfies aset of requirements (constraints).Let us define a general optimisation problem, and fix the notation and terminologythat will be used throughout the paper.Let x be an n-dimensional real vector, i.e. x = (x0, x1, . . . , xn) ∈ Rn. Then forsome real-valued functions (g0(x), . . . , gr(x)), we have the generic problem

min f(x)

subject to gi(x) ≤ 0, (1)

for i = 0, . . . , r, where f(x) is the objective function and the gi(x)’s are the con-straints.

A special case of (1) is convex programming, in which we assume that

• f : Rn → R is convex, and

• gi : Rn → R is convex ∀i.

The convexity property has a direct effect on the difficulty of the problem, asit implies that any local minimum (or maximum) must also be a global one[20].This is especially useful when dealing with nonlinear problems, in which the ob-jective function, the constraints or both are nonlinear.

The structure of this report is as follows. It comprises four main chapters, andaims to first provide the reader with an overview of the area interested, to then

1

Can semidefinite programming be a key approach to the pooling problem? 2

explore an original application of the theory presented. In the first two chapters,I treat a selected number of topics that constitute the core of the subject. Theintention is specifically to equip the reader with the necessary notions to approachthe work presented in later chapters. Some topics, though central to the field,are only marginal to the work carried out. In such occasions, in order to preservethe flow and the cohesion of the report, only an overview of the area is given,accompanied by references for further readings.Chapter 1 introduces the reader to semidefinite programming. We first famil-iarise with the key concepts and terminology, and discuss some technical toolsused for semidefinite programs. We then present an example of semidefinite pro-gramming relation for combinatorial optimisation.Chapter 2 focuses on the pooling problem. We consider the background andsituation in which it arises, and then proceed to provide a mathematical repre-sentation, and describe the difficulties intrinsic to the model. We conclude thechapter with a brief outline of the methods that have been used in the past.Chapter 3 aims to motivate the work in this project. We discuss the reasons forapplying semidefinite programming to the pooling problem, define the model usedin the paper and for the computational work, and then delineate the methodologyinvolved in constructing a first relaxation to the problem.Chapter 4 summarises the work implemented. It first presents the examplesthat were used throughout the computational work, then gives an account of theissues encountered and approaches taken to overcome them. Finally, we provideideas for further work, and strategies that we consider promising.

2

Chapter 1

Semidefinite Programming (SDP)

1.1 Introduction

This chapter provides the reader with an introduction to semidefinite program-ming and some of its applications. Much of the material covered is based onthe work of Boyd and Vandenberghe, who are leading authors in the field1. Anexhaustive review of the topic can be found in [30], while the reader should referto [31] for a more synthetic overview.

Semidefinite programming (or SDP) is an extension of linear programmingover the cone of positive semidefinite matrices, i.e. matrix inequalities substitutevector componentwise inequalities, and feasible variables need to be in the coneof positive semidefinite matrices, instead of the nonnegative orthant Rn

+. SDPis a relatively new branch of optimisation, and was one of the main areas of re-search in the field in the 1990’s. Experts from a variety of areas were especiallyattracted by the possibility of applying it to a range of problems in combinato-rial optimisation and control theory. The advances in interior-point algorithmsfurther increased the popularity of the field, providing an increasingly efficientmethod to reach a solution.

Let us start by defining a few mathematical objects, and setting the notation.Let X ∈ Sn, where Sn is the space of real symmetric n × n matrices. DefineC •X to be the inner product of the symmetric matrices C and X, i.e.:

C •X =n∑i=1

n∑j=1

CijXij.

Note that this is the same as writing C • S = TrCX, where TrCX is the traceof the matrix CX; this simply means that C(X) is a linear function of X.Furthermore, let X � 0 denote that X is positive semidefinite, i.e. zTXz ≥ 0 forall z ∈ Rn, z 6= 0.

1It is worth mentioning that Boyd is, with M. C. Grant, one of the creators of CVX, theoptimisation package used for the computational work in Chapter 4.

3


We can then define a semidefinite programming problem as follows:

min C •X (1.1a)

s.t. Ai •X = bi, for i = 1, . . . ,m, (1.1b)

X � 0, (1.1c)

where the Ai’s, C and X are all in Sn, and X is the unknown.In general, a matrix inequality X − Y � 0 can be denoted as X � Y , with bothexpressions stating that X − Y is positive semidefinite.

Before proceeding any further, it is worth noticing that semidefinite program-ming is a special case of convex optimisation, as both the objective function andthe constraints are convex. This is one of the reasons why, even though semidef-inite optimisation problems are much more general than linear programs, theyare not much harder to solve.

1.1.1 Duality

The duality theory for SDP programs has been thoroughly studied, and is fun-damental in the formulation of some of the methods used to solve this family ofproblems [32]. Below are the key concepts and notions from the area 2.

Recall that, for linear programs (LP), we have:

PRIMAL LP

min cTx

s. t. Ax = b, (1.2)

x ≥ 0,

DUAL LP

max bTy

s. t. ATy + s = c, (1.3)

y free, s ≥ 0,

where c, x ∈ Rn, b ∈ Rm, A ∈ Rm×n, y ∈ Rm and s ∈ Rm.

Now consider the SDP (1.1). Let A ∈ Sn, so that we can rewrite constraint(1.1b) as:

AX =

A1 •X...

Am •X

∈ Rm.

Let A∗ : Rm → Sn denote the adjoint operator to A, such that A∗ =m∑i=1

yiAi.

Recall the standard result from Linear Algebra:

(AX)Ty =m∑i=1

yi(Ai •X) =

(m∑i=1

yiAi

)•X.

2Proofs are omitted, but may be found in most of the references on semidefinite programmingprovided in the bibliography.

4


for X ∈ Sn, y ∈ Rm.Then, in a similar manner to before, we can define the primal and dual SDPproblems as follows:

PRIMAL SDP

min C •Xs.t. AX = b, (1.4)

X � 0,

DUAL SDP

max bTy

s.t. A∗y + S = C, (1.5)

S � 0,

where y ∈ Rm and S ∈ Sn.Note that the dual SDP can be expressed in a similar form to (1.1) as

max bTy

s.t.m∑i=1

yiAi + S = C, (1.6)

S � 0.

We can combine the two constraints in (1.5), using the first to derive an expression

for S, i.e. S = C − A∗y (or, equivalently, S = C −m∑i=1

yiAi). The alternative

formulation to (1.7) follows:

max bTy

s.t.m∑i=1

yiAi � C. (1.7)

We call the constraint in (1.7) a linear matrix inequality (or LMI).

We can define the strictly feasible regions for the primal (1.2) and the dual(1.3) respectively as:P0 = {X | AX = b,X � 0},D0 = {(y, S) | A∗y + S = C, S � 0}.Then, the primal-dual feasible set isF0 = {(X, y, S) | AX = b,A∗y + S = C, (X,S) � 0}.

As for the linear case, it is possible to define weak and strong duality forsemidefinite programs.

Theorem 1.1.1.1 (WEAK DUALITY). Let X be a feasible solution of the primalproblem (1.1) and (y, S) a feasible solution for the dual (1.5). Then the dualitygap is defined as

C •X − bTy = X • S ≥ 0. (1.8)

Note that if C • X = bTy, then X and (y, S) are optimal solutions for theprimal and the dual respectively, and XS = 0.

5


Theorem 1.1.1.2 (STRONG DUALITY). Let X and (y, S) be optimal solutionsfor the primal (1.1) and dual problem (1.5) respectively, and let z∗p and z∗d be theassociated finite optimal objective values. Then, if certain regularity conditionsare satisfied,

z∗p = z∗d, (1.9)

When theorem 1.1.1.2 holds, we have a zero duality gap.It is important to notice that convexity is one of the necessary conditions forstrong duality, and is indeed satisfied by all SDP problems.

Slater’s regularity conditionThere exist X ∈ P and S ∈ D such that X � 0 and S � 0.

Slater’s condition essentially states that z∗p and z∗d are strictly feasible.

If X and (y, S) are as in 1.1.1.2, and if z∗p and z∗d satisfy Slater’s condition, i.e.they are in the interior of the respective feasible region, then strong duality isguaranteed.Note that strict feasibility is often exploited to determine strong duality, as itis relatively easy to verify (especially compared to other regularity condisions),and it is a sufficient condition for strong duality to hold for convex optimisationproblems.

The next section considers the process of solving semidefinite programs. Whilea straightforward simplex method does not exist for this family of problems, mostalgorithms for interior-point methods have been adapted to handle them and arenow the preferred approach in the field.

1.2 Interior-point methods

Interior-point methods were introduced for LPs by Karmakar in a seminal paperpublished in 1984 [22], although some of the principles they rely on, such as theuse of barrier methods, have been around since the 1950s. In the 1990s, Al-izadeh and Nesterov where the first to implement interior point methods withinthe semidefinite programming framework[2][27].This section aims only to give a brief overview of the method and its merits. Formore details on the methodology and technical issues, [32] is a standard refer-ence for primal-dual interior-point methods. The reader might also find [3] ofinterest, for its specific focus on SDP and combinatorial optimisation, which isparticularly relevant to the content of this report. Most of the material in thissection is derived from [14] and [26]: the latter is a lecture series that provieds anexcellent introduction to the topic, and the reader should refer to it for a moredetailed discussion on polynomial time, and a more rigorous derivation of thenotions presented.

6


Let us go back to the general LP problem (1.2), and let P0 be its strictly feasi-ble set. In linear programming, an optimal solution x would lie on the boundaryof such a region[20].

The simplex method exploits such a char-acteristic to find a solution for LP’s, bystarting from an x0 and proceeding to theadjacent vertex that yields the best im-provement in the objective value. Interior-point methods find, instead, a central tra-jectory through the feasible region, in theattempt to approximate the central path,as show in the picture [14]. More specif-ically, primal-dual interior-point methodgenerate a sequence of points in F0, theprimal-dual feasible region, that will con-verge to an optimal solution x, maintaining both primal and dual feasibility whileaiming for complementarity.The basic idea of this approach is to compute a Newton step in each iterationwith respect to some auxiliary barrier problem that keeps the iterates inside thepositive semidefinite cone.

Let us now introduce the theory of interior point methods for semidefiniteprogramming.We start from a feasible point3, i.e. a point in the interior of the feasible region.The following step relies on the idea that an unconstrained convex optimisationproblem may be easier to solve, leading to the attempt to somehow incorporatethe nonnegativity constraint in the objective function. For this purpose, exploitthe concept of barrier, i.e. an interior penalty function f(x) that is smooth andconvex. This function needs to be defined on the interior of the feasible regionP0, and behave in a very specific manner, tending to infinity when it approachesthe boundary of P0.

Let us now proceed to illustrate how such a function is used for semidefiniteprograms. First, we can define the primal barrier SDP as follows:

min C •X + µf(X)

s.t. AX = b, (1.10)

for a given barrier parameter µ ≥ 0, which controls the relation between thebarrier term and the objective value of the original SDP.Note that the positive semidefinte constraint X � 0 has been dropped, while thebarrier term has been added to the objective function, essentially reproducing

3This is the case for feasible interior point methods. Note that finding such a feasiblepoint is often difficult, so a lot of research has focused on developing infeasible interior pointmethods. Currently, infeasible primal-dual interior point algorithm are broadly considered themost efficient interior point method available [15]

7


the effect of the constraint removed.

Logarithmic functions are amongst the most common types of barrier functionused, mainly due to the behaviour of log(x), which tends to negative infinity as xtends to zero. Let us define a logarithmic barrier function for the cone of positivesemidefinite matrices f : Sn×n+ → R,

f(X) =

{−ln det(X) if X � 0,

+∞ otherwise.(1.11)

Such a barrier function is convex and has the advantage, over other possible othercandidate function that may behave similarly in a neighbourhood of zero, of alsobeing self-concordant.Roughly speaking, if a function f : R→ R is self-concordant, then

|f ′′′(x)| ≤ 2f ′′(x)3/2. (1.12)

Such a feature is particularly important in the context of interior point method:Newton’s method exploits second order information to operate, and hence workswell for this class of barrier problems, which ensure fast convergence. Note that,since the logarithm is a strictly concave function on the positive real axis, thefunction f(X) = −ln det(X) is strictly convex. As a consequence, it attains itsoptimum in the interior of the feasible region P0. We use Newton Method to finda stationary point of the barrier problem, in order to determine an appropriatedirection in which to proceed. First, we rewrite the Lagrangian of (1.10)

Lµ(X, y) = C •X + µf(X)− yT (AX − b), (1.13)

with y ∈ Rm.We known that an optimal solution for the barrier problem is also a saddle pointof its Lagrangian. Such saddle points satisfy the Karush-Kuhn-Tucker (KKT)optimality conditions4. Now, let f(X) = −ln det(X) as suggested above, whichhas f ′(X) = −X−1. Furthermore, note that for a positive semidefinite X, theinverse X−1 is also positive semidefinite.Then, differentiating Lµ(X, y) with respect to its variables, we get the first orderoptimality condition (FOC) for a stationary point of Lµ:

5XLµ(X, y) = C − µX−1 −A∗y = 0, (1.14)

5yLµ(X, y) = b−AX = 0. (1.15)

Now, if we let S = µX−1, i.e. XS = µI, the first order optimality conditions can

4See [20] for a detailed introduction to the topic.

8


be rewritten as:

AX = b, (1.16)

A∗y + S = C, (1.17)

XS = µI, (1.18)

X,S � 0. (1.19)

Note that the first and the second conditions require primal and dual feasibilityrespectively. Furthermore, the third equation is the perturbed complementaritycondition (that is, the complementarity condition for the barrier problem, whilefor the original SDP we have a zero in the right hand side).

For a given µ > 0, the µ-centre is the unique solution (X, y, s) of the systemabove. The primal-dual central path, also called central trajectory, is a (continu-ous) path {(X, y, S) : µ > 0} determined by a family of such solutions.

We are interested in finding an approximate solution of (1.2), i.e. a solutionto

Fµ(X, y, S) =

AX − bA∗y + S − CXS − µe

= 0.

We can now use Newton’s method to solve this system of nonlinear equations.That is, we compute a step direction (4X,4y,4S) by solving

Fµ +∇Fµ · (X, y, S)T = 0,

where 4X,4 ∈ Sn×n, 4y ∈ Rm. We can accomplish this by finding a solution(4ξb,4ξC ,4ξµ) for the system A 0 0

0 A∗ Iµ(X−1 �X−1 0 I

4X4y4S

=

4ξb4ξC4ξµ

. (1.20)

In (1.20), we introduce the operator �: Sn×n to Sn×n, defined as

(P �Q)U = 1/2(PUQT +QUP T ), (1.21)

for matrices P,Q ∈ Sn×n. In particular, the choice of Q = P = X−1 and U = Icorresponds to the search direction (1.20).

Note that we will fix the barrier parameter µ and make only one Newton stepat a time towards the solution of the optimality conditions. In other words, wedo not solve the current FOC exactly. Instead, interior point algorithms focuson finding a suitable sequence {µk}, and iterate the step above until optimalityis reached. To begin with, a large value of µ is chosen, so to strengthen thebarrier term and ensure that the solution X in the interior of the cone of posi-tive semidefinite matrices, i.e. centrality is promoted. The parameter µ is then

9


gradually reduced, hence the influence of the logarithmic barrier is reduced. Thiscorresponds to moving the focus from centrality to optimality, in order to eventu-ally approach an optimal solution (which may be on the boundary of the feasibleregion).

It is worth mentioning some of the reasons that make them the approach ofchoice for solving a variety of linear and nonlinear optimisation problems [15] [31].First, they have low-degree polynomial worst-case complexity, that is, “the effortrequired to solve a semidefinite program to a given accuracy grows no faster than apolynomial of the problem size.”[31]. Furthermore, they are able to reach optimalsolutions in an almost constant number of iterations, almost independently fromthe size of the problem. Indeed, they are particularly competitive for very largeproblems if compared to other methods.

1.3 Applications

Semidefinite programming can be applied to a variety of fields. Examples includenon-convex quadratic optimisation, statistics (for instance, the minimisation ofregression), eigenvalues problems, and structural design. Should the reader beinterested in a more detailed discussion of the above applications, as well as somemore, [30] covers systematically the theory and notation in each case, while alsoproviding practical examples. Furthermore, the CVX website [8] has an extensiveexample section that might appeal to an audience with a deeper curiosity andunderstanding for computational optimisation.

Control theory and combinatorial optimisation are currently the two mostattractive applications of SDP. Nonetheless, a discussion of control theory is out-side the scope of the project. Combinatorial optimisation is, instead, of particularinterest as, aside the popularity of this field, the approach shown here can alsobe applied to global optimisation problems. Indeed, the pooling problem fallsunder this category, and the theory and procedures presented will be later usedin Chapter 3 and 4.

1.3.1 Combinatorial optimisation

A combinatorial optimisation problem is an optimisation problem in which thefeasible set is discrete (instead of continuous). An increasingly popular approachin solving this family of problems is to derive their semidefinite relaxation.

Let us consider the following quadratic program (QP):

min f0(x)

s.t. fi(x) ≤ 0, ∀i, (1.22)

where fi(x) = xTAix+ 2bTi x+ ci, i = 1, . . . , L.Note that, if the matrices Ai are indefinite, problem (1.22) is a nonconvex opti-

10


misation problem, and finding a solution suddenly becomes much harder. Thisis, indeed, true for most real-life situations, and represents the focus of this report.

Often, it is desirable to compute lower bounds on the optimal value of the QP(1.22), and do it as cheaply and efficiently as possible. One may try to obtainsuch a bound by solving the semidefinite program

max t

s.t.

[A0 b0bT0 c0 − t

]+ τ1

[A1 b1bT1 c1

]+ · · ·+ τL

[AL bLbTL cL

]� 0, (1.23)

τi ≥ 0, i = 1, . . . , L,

with variables t and τi.Let us now show that such a semidefinite program provides a lower bound for(1.22). First, let x be such that

fi(x) =

[x1

]T [Ai bibTi ci

]︸︷︷︸

F

[x1

]≤ 0,

for i = 1, . . . , L, i.e. x satisfies the constraints in the nonconvex problem (1.22).Assume also that t and τ1, . . . , τL satisfy the constraints in the semidefinite pro-gram (1.23).Then, for any x,

0 ≤[x1

]T ([A0 b0bT0 c0 − t

]+ τ1

[A1 b1bT1 c1

]+ · · ·+ τL

[AL bLbTL cL

])[x1

],

and using F to express all fi(x) in (1.22),

= f0(x)− t+ τ1f1(x) + · · ·+ τLfL(x)

≤ f0(x)− t.

Rearranging the last line, it follows that t ≤ f0(x) for every feasible x in (1.22).In particular, we can then construct an x from the optimal solution of the QP(1.22), so that we just obtained a lower bound t for the problem by solving itsSDP relaxation (1.23).

In general, most combinatorial problems can be manipulated to produce arelaxed program such as the one in (1.23), or in it’s equivalent SDP Shor’s relax-ation [31]

min TrXA0 + 2bT0 x+ c0

s.t. TrXAi + 2bTi x+ ci ≤ 0, i = 1, . . . , L (1.24)[X xxT 1

]� 0,

11


where the variables are X = XT ∈ Rk×k (i.e. X ∈ Sk) and x ∈ Rk. In later chap-ters, we will be applying a relaxation analogous to (1.24) to the pooling problem.

Generally, a key step of the approach presented is to notice that the constraint[X xxT 1

]� 0, (1.25)

is equivalent to X � xxT .Thus, one can draw a direct link between the original problem (1.22) and thesemidefinite program (1.24), which can be seen as its relaxation.First, go back to (1.22) and rewrite it using (1.25) to get

min TrXA0 + 2bT0 + c0

s.t. TrXAi + 2bTi x+ ci ≤ 0, i = 1, . . . , L, (1.26)

X = xxT .

Now, comparing (1.24) to (1.26), observe that the replacement of the constraintX = xxT by its relaxation X � xxT is the only difference between the twoproblems. In fact, X = xxT is a symmetric rank-1 positive semidefinite matrix,and could equivalently be expressed with these two conditions separately, i.e.

X = xxT ⇐⇒

{X � xxT and

rank(X) = 1.

Hence, by imposing that the matrix on the left-hand side of (1.25) is rank one(which is a nonconvex contraint), we could obtain the original problem (1.26)back. This simple relaxation, though, is crucial to the process, as it bears theimportant advantage that the new constraint is convex.

Let us illustrate this process with an example [31]. Consider the NP-hardquadratic program

min xTAx+ 2bTx (1.27)

s.t. x2i = 1, i = 1, . . . , k. (1.28)

First, note that we can rewrite the constraint xi ∈ {−1, 1} as the quadraticequality x2 = 1. This, in turn, can be expressed as two quadratic inequalities,namely x2 ≤ 1 and x2 ≥ 1. Next, we can write the following SDP relaxation ofthe problem, with form similar to (1.26):

min TrXA+ 2bT

s.t.Xii = 1, i = 1, . . . , k (1.29)[X xxT 1

]� 0,

where we have variables X = XT and x. By solving (1.29), we then obtain a

12


lower bound for the optimal value of (1.27).Note that the matrix in constraint (1.25) is, by convention, taken to be posi-tive semidefinite, but the direction of the LMI can be picked arbitrarily (thoughconsistently within a problem).

13

Chapter 2

The Pooling Problem

The diet problem, or Stigler diet, was initially formalised by Stigler in 1945, andmodelled and solved by Dantzig as the first LP a few years later [9]. As its namesuggests, it addresses the issue of constructing the dietary program that bestsatisfies some given nutritional needs.Such a model can be adapted to a variety of contexts; essentially, it can beinterpreted as the search for the best combination of a set of raw materials to formproducts that satisfy some given requirements. The easiest type of diet problem isthe blending problem, in which the raw materials themselves are directly combinedto form products. A more advanced and convoluted extension is the poolingproblem, first formulated by Haverly in 1978 [18], where a set of intermediatemixing bins is introduced. Let us examine the latter category in the followingsection, to then focus on its mathematical formulation, and discuss some of themethods employed so far.

2.1 Overview

The pooling problem arises recurrently in the petroleum industry, and addressesfundamental issues underlying the refinery process.1

In a refinery, different types of crude oils are distilled to produce a number of in-termediate mixers, which will then be use to blend the desired final products. Itis important to note that crude oils can differ considerably, as their compositionand properties are strongly influenced by their location of origin. After poolingtwo or more of these raw materials (let us refer to them as raws), they are nolonger present in their original form. Instead, they essentially create some in-termediate products. Although such mixes will have attributes and compositionrelated to those of the original oils, some form of dilution may also take place,inducing new property values. These blends are, once again, pooled to obtain thedesired products, which must meet quality requirements set by the costumers,normally expressed as value per unit amount of some components.

The objective is to maximise the total sales or, equivalently, to minimise the

1The problem is applicable to a variety of other engineering and environmental issues, suchas supply-chain operations, waterwaste treatment, etc.

14


production costs. Observe that the price of a final product is not known, as itis dependent on its associated property values, which are, in turn, related to theintermediate blends.

The following diagram illustrates an instance of the problem: in this example,four crude oils (RAW) are mixed in three pool tanks (MIX) to form the finalproducts (PROD). Observe that some producers also allow straights, that is, one

Figure 2.1: A fictitious pooling problem.

or more original crude oils can flow directly into the products.

The pooling problem can be translated into a nonlinear programming model,as illustrated in the following section. We will then examine the issues that arisefrom such a formulation, which allows for multiple locally optimal solutions, andrequires good initial values to avoid convergence to a poor local optimum [20].

2.2 Parametrisation

The pooling problem can be seen as a blending problem extended both in terms ofsize, as it involves blending multiple products, and difficulty. In fact, without themixing bins, the problem would be a simple LP. Instead, pooling the raw materialsin the intermediate tanks, nonlinearity is introduced through bilinear constraints.

Let us analyse conceptually what the problem entails. A given amount of rawmaterial is provided, each with specific properties (chemical content, viscosity,density, etc.). A number of pooling tanks are available to produce mixers. Then,final products are formed, which have to comply with quality requirements set

15


by the customers. This process is governed by two main laws, which control thefinal step between the pooling tanks and the products: the mass conservationand the value conservation law (as labelled by Nishi [28], whose paper inspiredthe work in this project).

Let us consider the simplest possible scenario, where only two mixers (denotedas 1 and 2) are blended into a single final product (3). Define the amount ofproduct as x, and let p be some property we are interested in (given as value perunit amount).We then need to ensure the mass balance is maintained, and can achieve thiswith the linear equality

x3 = x1 + x2. (2.1)

At the same time, the value of the final product depends on the property of theintermediate ones, as described by the equality

p3x3 = p1x1 + p2x2. (2.2)

This second equation is a source of nonlinearity when both p3 and x3 are un-knowns.We can now proceed to formally construct the mathematical representation of theproblem. Note that the model used is based on the formulation by Grothey et al.[16], where the flow variables are defined in terms of flow proportions. Althoughthis differs from models using total flows, they both have the same global optimalsolution.Without affecting the relevance of the work presented, we are free to opt for theversion in [16], as this will allow comparison of results in later sections.

First, consider a qualitative version of the model, with references to the cor-responding mathematical constraints.

Minimise: total costs of production (2.3a),Such that:

for each mix

• the nutrient concentration depends on the fraction of raws presentand the nutrient content of each of those (2.3b),

• the cost per unit is defined by the proportion of raws used, andtheir cost(2.3c),

• the portion of raw material sums up to one (2.3d),

• no negative flow from the raws is allowed 2(2.3e);

2The flow proportion values may be negative, if the model allows for drying, that is, ”somemixing bins do not have the normal role of mixing ingredients, but instead dry the mix theyare given, by reducing the amount of the water raw material.” [17]

16


for each product

• the property value, represented by the nutrient concentration,depends on the property of the mixes used, and must fall withinan interval imposed by the customers (2.3f),

• the proportional flows of mixes add up to one (2.3g),

• no negative proportion of mixes is pooled (2.3h).

Note that optimisation of production costs is accomplished by minimising the sumof the cost for each product. That is, for each of them we consider the amount re-quired, the fraction of raw materials in the mixes used, and the price of the latter.

A mathematical model follows naturally from the above observations.First, let us fix the notation. The structural parameter, giving the dimensions ofthe problem, are as follows:

R = no. of raw materials −→ index iM = no. of mixers −→ index jD = no. of products −→ index kN = no. of nutrients −→ index l

Table 2.1 defines all the other symbols used in the model, making a clear distinc-tion between known parameters and unknown variables.

Parametersril unit nutrient content l in raw material ici cost of raw material i per unit

dlk, dlk bounds in unit nutrient content i in product ktk units of product k demanded

Variablesµij fraction of material i in premix jmjl unit nutrient content l in premix jcmj unit price of premix jλjk fraction of premix j in product k

Table 2.1: The pooling problem model notation: parameters and variables.

As shown in the table above, for each of the raws we have data pertaining thecost per unit and their nutrient concentrations. We also know the mass andnutrient concentrations for mixes, and the mass of final blends. Finally, we aregiven bounds for the required proportion of nutrient content for the products.We are, on the other hand, seeking information about the mixes, namely theircost per unit, nutrient concentration, and raw composition. Furthermore, we wishto determine the composition of products in terms of the intermediate blends.

17


The resulting mathematical model is given below.

minD∑k=1

tk

M∑j=1

λjkcmj (2.3a)

s.t.R∑i=1

µijril = mjl, ∀j, l, (2.3b)

R∑i=1

µijci = cmj , ∀j, (2.3c)

R∑i=1

µij = 1, ∀j, (2.3d)

µij ≥ 0, ∀i, j, (2.3e)

dlk ≤M∑j=1

λjkmjl ≤ dlk, ∀k, l, (2.3f)

M∑j=1

λjk = 1, ∀k, (2.3g)

λjk ≥ 0, ∀j, k. (2.3h)

Observe that the first four constraints govern the properties of the intermedi-ate blends, whilst the final three address the demands to be satisfied.

2.3 Different formulations

As mentioned in the previous section, several formulations can be used to modelthe pooling problem. Independently from the formulation of choice, we may talkof generalised pooling problem (GPP) to indicates that flow is allowed betweenpooling tanks. This is the case for many real life problems, but most of the re-search so far has focused on the standard version.Before presenting the two main formulations available, recall that the mathemat-ical models obtained, thought different in form, are equivalent, and lead to thesame global optima.

• p-formulation: also called the total flow model, it’s a natural way to modelpooling problems, and was used by Haverly to formulate the first instance inthe literature [18]. It works with total flows volumes, and defines variablesfor the total flows between any two bins.

• q-formulation: introduced by Ben-Tal, Eiger and Greshovitz in 1994, itreplaces the total flows from raws to mixes with flow proportion variables.Such variables measure, for each mix, the proportion of a input from eachof the raws.

18


Both formulations are originally intended for standard problems, but may beadapted to generalised pooling problems if desired. In this project, the extendedq-formulation is chosen, with flow proportion variables replacing the measuresof total flow of raws and nutrients (both for the inflow and outflow of bins). Thisformulation was proposed by Grothey [16] for standard pooling problems, whilean extension to GPPs is available in [17].

Diagram 2.2 summarises the formulations described.

Figure 2.2: Pooling problem formulations.

2.4 Geometric interpretation

This section briefly introduces a graphical interpretation of the pooling problem,which was used to construct the example Grothey1 discussed in Chapter 4.Grothey et al.[16] observe that fixing the mix compositions reduces the standardpooling problem to an linear program, that can be decomposed into linear sub-problems (M LPs to form the mixes, and P to form the products).

Figure 2.3: Nutrient space for R,D,N = 2,M = 3.

19


Following the approach they used, we can give a geometric view of the prob-lem. In fact, we can imagine that nutrients span a multidimentional space, and,when we fix the mix compositions, we are essentially placing each pooling tankwithin such a space.This process is illustated in diagram 2.3, where we are considering a small prob-lem, with only two nutrients, three raws, two mixes and two products. Note thatthe nutrients span the horizontal and vertical axes, while the raws determine theraw convex hull, i.e. the smallest convex shape containing all the products. Notethat, for a solution to be feasible, all mixes and products must lie inside thisregion.

Figure 2.4: The graphical representation of thesolution in [18]

It is worth noticing that thisoutlook is similar to the graph-ical representation proposedby Haverly in his paper [18].There, he depicts the multi-dimensional space spanned bythe proportional flow from thedifferent raws, in order to thenplace the single mix present hismodel. His results are shownin Figure 2.4, where the dot-ted lines represent gradients ofequal costs.

2.5 What’s so difficult?

It was mentioned in earlier sections that the pooling problem is potentially nonlin-ear. In fact, note that constraint (2.3f) is analogous to equation (2.2): it containsbilinear terms, where a flow proportion and a nutrient concentration variable aremultiplied together. Similarly, the price for the mixes is not given, as it dependson their unknown composition. Hence, the objective value (2.3a) in this formu-lation is nonlinear as well.

Figure 2.5: Minima for linear, nonlinear, convex and nonconvex functions.[17]

20


Solving a nonlinear programming problem is, by its very nature, more diffi-cult than for the linear case. The simplex method is not applicable to such afamily of problems, and more advanced and complicated algorithms need to beused. Furthermore, not only it is harder to obtain a solution, but there is alsono guarantee that the one found is globally optimal. In fact, while all linearfunctions are convex, nonlinear functions may be nonconvex. As mentioned inthe introduction, nonconvex functions can have multiple local minima, the bestof which is defined to be the global minimum. This is shown in Figure 2.5.

Besides potential nonlinearity, pooling problems are characterised by noncon-vexity (apart from trivial instances)[4]. As a consequence, for real scale poolingproblems, computing the global optimum becomes often unrealistic. Instead, oneoften strives for methods that obtain a good enough solution in an efficient man-ner.

Let us consider, in the following section, some of the techniques that havebeen used to solve the pooling problem.

2.6 Approaches so far

A variety of methods have been suggested and developed. This section is a brief,non-comprehensive overview of the main approaches proposed, while [1], [4], and[25] are excellent sources for a more detailed analysis (with the latter focusingespecially on the more recent developments).

Most methods for convex optimisation are both global and fast, that is, theyconverge quickly to the global optimum for the problem. For nonconvex optimi-sation, one generally needs to waive one of these two requirements. In fact, localoptimisation methods have often fast convergence, but may not find a global so-lution, nor can they identify it as such when they do. On the other hand, globaloptimisation methods lead to a global optima, but often converge very slowly.

The techniques used to approach the pooling problem can, also, be dividedin these two categories. More preceisely, sequential linear programming (SLP) isthe main local optimisation method, while branch-and-bound is the most popularglobal optimisation one. All the other methods presented below are extensionsof these two techniques, or complementary procedures that perform within theirframework.

Recursion methods were the first to be employed, with Haverly himself propos-ing a two-step iterative algorithm after formulating the problem in 1978[18]. Hisprocess aimed to estimate and, then, fix the properties of the intermediate prod-ucts, referred to as complicating variables, to then proceed to solve the resultinglinear program. Such complicating variables were then updated with the valuesestimated, until the resulting qualities coincided with the estimates. As men-tioned previously, because of the intrinsic characteristics of the problem, this

21


process may not lead to a global optimum.Such an approach initiated the idea of sequential linear programming for thepooling problem, first published in 1961 by Griffith and Stewart of Shell Oil.Their procedure consists of using a first-order Taylor expansion to replace bilin-ear terms, in order to obtain a suitable direction in which to move for the nextiteration.

Branch-and-bound algorithms can be applied to bilinear problems (BLP).Through controlled enumeration and relaxation, they divide the original feasi-ble region into smaller subsets. For each of those, lower and upper bounds ofthe objective function are then computed. Depending on such bounds, optimal-ity or feasibility arguments are drawn to select some of the subsets for furtherrefinement, and exclude others. It is worth noticing that the efficacy of branch-and-bound algorithms depends on the tightness of the bounds used. Althoughthey can indeed reach a global optimum, or a solution as close to it as desired,they may converge very slowly. In fact, they essentially reduce to enumerationin the worst case, with associated exponential convergence rate. Consequently, itbecomes essential to develop bounding methods that give tight bounds, in orderto ensure good performance.

The following are only some of the main techniques developed to implementor complement the methods above.

Analysis may be used to better understand the behaviour of the problem in-vestigated, and improve the algorithm used. Lodwick first introduced in 1992the idea of performing a sensitivity and feasibility analysis on the bilinear models(BLP), so to determine which constraints are tight, and whether some may beredundant at optimality.

Decomposition methods, such as the generalised Bender’s algorithm, are avery common approach for this problem, and rely on the observation that, byfixing the values of certain variables, a difficult problem may be simplified con-siderably. For instance, one may use an approximation of the feasible region foreach of the mixes to fix their composition, making the original problem linear [16].

Lagrangian relaxations techniques are often used as part of other methods,such as the global optimisation algorithm (GOP). They stem from the observationthat otherwise easy problems are often complicated by the presence of specificconstraints. Lagrangian relaxation replaces such constraints with penalty termsin the objective function, each consisting of the amount of violation of the con-straints multiplied with the associated dual variables (Lagrange multipliers). TheLagrangian subproblem is usually easier to solve than to the original problem,and provides a lower bound on its global optimum in the case of minimisation.

Let us apply the notions covered in the first two chapters and present themodel developed in this project.

22

Chapter 3

A Semidefinite Programmingrelaxation for the PoolingProblem

As mentioned in the previous chapter, the presence of mixing bins induces non-linearity in the pooling problem, and, on top of that, nonconvexity characterisesnontrivial cases.Section 2.5 discussed the main consequences of nonlinearity and nonconvexity; itwas observed that, for many practical scale pooling problems, obtaining a globaloptimum is unrealistic, so that one often aims for a compromise between qualityand computational cost of a solution.

Much research was spurred by the pursuit of methods that could provide globaloptima for this types of problems, or, alternatively, as tight and efficient boundsas possible. This project was inspired, in particular, by the efforts presentedin [1], [23] and [28]. The following section gives an overview of the work theysummarise, and proceeds to provide the key motivations for this project.

3.1 Theoretical contributions and proposed de-

velopments

We know, by now, that finding the optima of a pooling problem entails solving anonlinear optimisation problem that has multiple local minima.In 1999, Adhya et al. [1] introduce a new Lagrangian relaxation approach, so toprovide lower bounds for the solution. They test their algorithm on a variety ofproblems from the literature and on four new particularly difficult ones1. Thelower bounds obtained through their approach prove to be tighter than thosefound with the standard methods applied before. In reporting their results, theyattribute the improvement in the bounds obtained to their innovative choice ofconstraints to relax. In fact, in their formulation, they choose to relax the orig-inal constraints in a way that takes into consideration the characteristics of the

1Among these 13 tests are the examples that will be analysed in the next chapter.

23


problem, although it doesn’t necessarily lead to easy subproblems.

In 2010, Nishi [28] proposes to apply the semidefinite programming relaxationfor the pooling problem, formulated as a mixed-integer network flow model. Hethen takes this approach a step further, trying to obtain a solution for the originalprogram, rather than just a bound for it. First, he reformulates the problem asa continuous polynomial optimisation problem (POP), i.e. a nonlinear programwith polynomial objective function and constraints. Then he observes that ap-plying the SDP relaxation may cause the size of the SDP to become very large,while the solution of the SDP is obviously not necessarily feasible for the originalproblem. With respect to the problem size, he proposes a formulation that doesnot include the binary variables. Such a strategy, though, causes an excessiverelaxation of the feasible region of the original problem. To correct this, he sug-gests using valid inequalities, i.e. some redundant constraints of the original POPthat can reduce the feasible region of the SDP, and hence may lead to better asolution (and, hopefully, to one that is indeed feasible for the original problemtoo). He finally presents some numerical experiments, and shows that the formu-lations proposed may lead to better solutions compared to those obtained withthe POP model (before the relaxation), although at a high computational cost.

More recently (2012), Lavaei et al. [23] apply Lagrangian relaxation to theoptimal power flow (OPF) problem, which is, much like the pooling problem, non-convex and generally hard to solve. They propose solving the Lagrangian dualproblem instead of the original OPF problem, which is a convex semidefinite pro-gram. Then, they provide a necessary and sufficient condition that guaranteesthat the duality gap is zero, which is satisfied in all benchmark systems theyconsider in their study. They finally proceed to propose an alternative sufficientcondition for zero-duality-gap that is easier to verify. The obvious consequence ofsuch conditions being satisfied is that, when the duality gap is zero, it is possibleto use a dual optimal solution of the OPF to recover a globally optimal solution.

This projects aims to apply a semidefinite programming relaxation to thepooling problem, modelled using the extended q-formulation. Note that the pro-cess presented resembles the approach to the combinatorial optimisation problem1.27 in Section 1.3.1. Once the relaxation is performed, we look for redundantconstraints, analogously to Nishi[28]. In this case, though, the model is not dy-namic, while his formulation takes into account the time frame of the poolingprocess, and some of the extra constraints proposed are valid only in that con-text. Furthermore, he proposes a rather elaborate process, that requires multiplereformulations and relaxations. The purpose of this project is to see whether asimpler approach can be of value, and lead to tight enough bounds. Furthermore,we try to see whether the arguments of the optimal solutions for the relaxationmay be feasible for the original problem. In that case, they may in fact be usedto obtain an upper bound for the optimal value of the latter.

Let us now describe step-by-step the process conceived.

24


3.2 Model formulation

Consider the model (2.3) presented in Section 2.2.We have identified the objective function (2.3a) and the bounds on the nutrientcontent of products (2.3f) as the sources of nonlinearity.

First note that (2.3f) can be split into the following two constraints:

M∑j=1

λjkmjl − dlk ≥ 0, (3.1a)

dlk −M∑j=1

λjkmjl ≥ 0, (3.1b)

for all k, l.

Let us start with the objective functionD∑k=1

tkM∑j=1

λjkcmj .

Ignore, for the moment, the multiplicative factor tk, and fix the index k (in fact,we will see that k outer product matrices will be created), as the second sumdoes not depend on it. The second summation term consists of some of the outerproducts between λ’s and cm’s. Let x ∈ Rk, from Linear Algebra the outer prod-uct of x with itself is defined to be

xxT =

x1x2...xk

[x1 x2 . . . xk]

=

x1x1 x1x2 . . . x1xkx2x1 x2x2 . . . x2xk

......

. . ....

xkx1 xkx2 . . . xkxk

. (3.2)

Note that the resulting matrix is, by construction, symmetric. Now, let us con-sider how to replicate the summation above. First, define

x :=

[λ:,kcm

],

where λ:,k selects column k of the matrix (λj,k). We can then use the outer prod-uct xxT to obtain all possible combinations of λ:,k and cm with themselves andwith each other.It is worth considering an easy example; this will help us visualise what is goingon, and how to proceed to the next step. Pick k = 1 and consider the case wherethere are only M = 2 mixing bins. Then, we have:

[λ:,1cm

] [λ:,1cm

]>=

λ1,1λ2,1cm1cm2

[λ1,1 λ2,1 cm1 cm2]

=

(λ1,1)

2 λ1,1λ2,1 λ1,1cm1 λ1,1c

m2

λ2,1λ1,1 (λ2,1)2 λ2,1c

m1 λ2,1c

m2

cm1 λ1,1 cm1 λ2,1 (cm1 )2 cm1 cm2

cm2 λ1,1 cm2 λ2,1 cm2 cm1 (cm2 )2

. (3.3)

25


Let X1 be the matrix obtained in (3.3). Note that, for M = 2 and picking productk = 1, then the inner summation in the objective value would be

2∑j=1

λj,1cmj = λ1,1c

m1 + λ2,1c

m2 ,

that is, half of the sum of the highlighted entries in X1. We then need to finda way to pick the appropriate entries in X1, and sum them. First, note that wecan multiply X1 with the block matrix A ∈ R4×4,

A :=

0 0 1/2 00 0 0 1/2

1/2 0 0 00 1/2 0 0

,and obtain

(λ1,1)2 λ1,1λ2,1 λ1,1c

m1 λ1,1c

m2

λ2,1λ1,1 (λ2,1)2 λ2,1c

m1 λ2,1c

m2

cm1 λ1,1 cm1 λ2,1 (cm1 )2 cm1 cm2

cm2 λ1,1 cm2 λ2,1 cm2 cm1 (cm2 )2

0 0 1/2 00 0 0 1/2

1/2 0 0 00 1/2 0 0

= 12

λ1,1c

m1 0 0 0

0 λ2,1cm2 0 0

0 0 λ1,1cm1 0

0 0 0 λ2,1cm2

.

Finally, if we compute the trace of the resulting matrix, we get the desired result

Tr(X1A) = λ1,1cm1 + λ2,1c

m2 .

We can now generalise the above process for all products k = 1, . . . , D, and foran arbitrary number M of mixing bins, by constructing D matrices

Xk :=

[λ:,kcm

] [λ:,kcm

]>, (3.4)

and defining A ∈ R2M×2M to be the block matrix

A :=

[0 diag(1/2)

diag(1/2) 0

], (3.5)

where diag(1/2) indicates an M ×M diagonal matrix with diagonal entries equalto 1/2.The objective function (2.3a) can then be replaced by

minD∑k=1

tk Tr(XkA),

under the condition that constraint (3.4) is satisfied.

26


The same approach can be taken for the bounds (3.1). In this case, we defineD ×N matrices Ykl as follows:

Ykl :=

[λ:,km:,l

] [λ:,km:,l

]>. (3.6)

Constraints (3.1) can then be reformulated as

Tr(YklA)− dlk ≥ 0, (3.7a)

dlk −Tr(YklA) ≥ 0, (3.7b)

with the added constraint (3.6).Note that we are using matrix A again, as the Ykl’ and Xk’s have the same size.We can now reformulate model (2.3) in the same form of (1.26), to obtain thefollowing program:

minD∑k=1

tk Tr(XkA) (3.8a)

s.t.R∑i=1


R∑i=1


R∑i=1

µij = 1, ∀j, (3.8d)

µij ≥ 0, ∀i, j, (3.8e)

M∑j=1

λjk = 1, ∀k, (3.8f)

λjk ≥ 0, ∀j, k, (3.8g)

Tr(YklA)− dlk ≥ 0, ∀k, l, (3.8h)

dlk −Tr(YklA) ≥ 0, ∀k, l, (3.8i)

Xk =

[λ:,kcm

] [λ:,kcm

]>, ∀k, (3.8j)

Ykl =

[λ:,km:,l

] [λ:,km:,l

]>, ∀k, l, (3.8k)

with Xk, Ykl ∈ S2M×2M . Before going any further, recall from Section 1.3.1 that,if rank(X) = 1, an equality X = xxT can be rewritten as X � xxT , which, inturn, is equivalent to [

X xxT 1

]� 0.

27


Nevertheless, as we mentioned in before, the rank constraint is nonconvex.Use such a correspondence to reformulate constraints (3.8j) and (3.8k). Then,removing the rank constraints for all Xk’s and Ykl’s, we are left with a semidefi-nite program, or, more precisely, with the SDP relaxation of the original problem(2.3).Observe that, up until the removal of the rank constraint, the original problemand its feasible region had only been reformulated, but were still unchanged incontent.

The proposed SDP relaxation of the pooling problem is presented below.

minD∑k=1

tk Tr(XkA) (3.9a)

s.t.R∑i=1


R∑i=1


R∑i=1

µij = 1, ∀j, (3.9d)

M∑j=1

λjk = 1, ∀k, (3.9e)

µij ≥ 0, ∀i, j, (3.9f)

λjk ≥ 0, ∀j, k, (3.9g)

Tr(YklA)− dlk ≥ 0, ∀k, l, (3.9h)

dlk −Tr(YklA) ≥ 0, ∀k, l, (3.9i) Xk

[λ:,kcm

][λ:,k cm

]1

� 0, ∀k, (3.9j)

Ykl

[λ:,km:,l

][λ:,k m:,l

]1

� 0, ∀k, l. (3.9k)

The next section will provide a generic algorithm for the above relaxation,which could be implemented in a programming language of choice (MATLAB7.11was used for the computational work presented).

Note that a lot of the work behind this project focused on formulating validinequalities, so to reduce the feasible region of the proposed SDP relaxation (3.9).An account of such efforts is presented in the next chapter, more specifically inSection 4.2.

28


3.3 The proposed algorithm

ALGORITHM

STEP 1 INITIALISE THE PROBLEM

Set, from the data:

• the dimensions R (raws), M (mixes), D (products), N(nutrients),

• (ril): the raws unit nutrient content,

• c: the cost of raws ,

• (dlk), (dlk): the nutrient content bounds for products,

• t: the products demand.

STEP 2 SOLVE THE SDP RELAXATION

Using A as in (3.5), solve the SDP relaxation to the problem.

This returns a solution with objective value z∗d and variables

λ, µ, m, cm.

minimise sum tk(XkA) for k = 1, 2, . . . , Dsubject to

µT r == m

µT c == cm

for j = 1, 2, . . . ,M

sum(µ(:, j)) == 1

for i = 1, 2, . . . , R

µ(i, j) ≥ 0

end for

end for

for k = 1, 2, . . . , D

sum(λ(:, k)) == 1

for j = 1, 2, . . . ,M

λ(j, k) ≥ 0

end for

end for

for l = 1, 2, . . . , N

for j = 1, 2, . . . ,M

µ(:, j)T r(:, l) == m(j, l)

end for

end for

for k = 1, 2, . . . , D

for l = 1, 2, . . . , N

Tr(Yk,lA)− dl(l, k) ≥ 0

du(l, k)− Tr(Yk,lA) ≥ 0

end for

end for

29


for k = 1, 2, . . . , D

for l = 1, 2, . . . , N Yk,l

[λ(:, k)m(:, l)

][λ(:, k) m(:, l)

]1

� 0

Xk

[λ(:, k)cm

][λ(:, k) cm

]1

� 0

end for

end for

STEP 2bis SDP RELAXATION & VALID INEQUALITIES

Step 2bis may be performed instead of or after step 2.

Some valid inequalities are formulated and added to theSDP to tighten the feasible region (see 4.4 for examples).

As for Step 2, we obtain an objective value z∗d and values for

the previously unknown variables λ, µ, m, cm.

STEP 3 RETRIEVE BOUND(S) AND SOLUTION(S)

The solution of the SDP relaxation yields a lower bound z∗don the optimum of the original problem, z∗p .

Check if the solution found is feasible for the original

problem, i.e. do λ, µ, m, cm satisfy the constraints for (3.9)?

YES: λ, µ, m, cm are feasible NO: λ, µ, m, cm are infeasible

• z∗d is a lower bound to

the optimal value of (3.9);

• compute zp from

(2.3a) using λ, µ, m, cm;

• zp is an upper bound to


• consider adding valid

inequalities and going back

to Step 2bis to improve

bounds.

• z∗d is a lower bound to


• consider adding valid

inequalities and going

back to Step 2bis to

improve bounds.

NOTE: one may only be interested in finding a lower bound for the optimum of(2.3). In that case, Step 2bis can be repeated to improve the tightnessof the results attained. Nevertheless, without an upper bound (or a localoptimum obtained with some other method, acting, in fact, as an upperbound), it may be hard to determine the quality of the bound obtained,hence the idea of checking the feasibility of solution.

30

Chapter 4

Examples, results and furtherwork

In Chapter 2 the pooling problem was introduced, and modelled in the math-ematical form (2.3). A SDP relaxation approach was presented in Chapter 3,with the general algorithm laid out in Section 3.3. This chapter expands on thenotion of valid inequalities, which were only mentioned in Section 3.3 withoutproviding any details. Here I document the computational work done, with anoverview of the examples used, and propose a specific set of new inequalities. Ialso illustrate the motivations and observations they stem from, together withthe results obtained and some suggestions for potential improvements.

The algorithm was coded in MATLAB 7.11, using CVX 2.0 [8] for the SDPrelaxation problem. This is a software package for convex optimisation that uses,in turn, specific solvers to perform. For the tests presented, SeDuMi and SDPT3were used, although, in general, no difference was observed in the results obtainedin the two cases. Note that by default, these solvers use primal-dual interior pointmethods (see Section 1.2) to obtain a solution.

4.1 Examples

This project aims at exploring the possibility of applying an SDP relaxation ap-proach to the pooling problem, and considers the use of extra inequalities totighten the feasible region, and obtain better bounds. For this purpose, it is use-ful to run some numerical tests. This is not only a chance to verify the effectsof the extensions proposed, but can be a valuable operation in itself. Computerprogramming is a two-way process: one needs to have a very good understandingof the problem before being able to represent it into computational form. At thesame time, this translation procedure can be a source of insight into the problem,as it sometimes requires looking at it from a different perspective.

Tests were run on four main problems, the dimensions of which are shown inTable 4.1. The first three are adaptations of the tests presented in [1] (Foulds2,as the name suggests, was collected from [12], while Example1 and Example3were constructed by Adhya and Sahinidis). Grothey1 was, instead, specifically

31


formulated during this project.

R M N D

Example1 6 4 1 4Example3 5 2 4 2Foulds2 8 3 6 4

Grothey1 5 3 2 4

Table 4.1: Number of raws (R), mixes (M), nutrients (N) and products (D) ofthe examples used.

Recall that, when the mathematical model was presented in Section 2.2, thep-formulation presented by Grothey et. al [16] was chosen. Such a choice allowsfor comparison with their results, and hence to comment on the quality of thebounds obtained with this algorithm.

Below is the diagram representations of Grothey1, so to visualise the infor-mation in the table. In the problem, five raw materials, each with two nutrients,are used to blend four products via three intermediate mixes. 1

Figure 4.1: Diagram for Grothey1.

4.2 Tests and progress

This section studies the SDP model to identify opportunities to introduce validinequalities. Although this approach is inspired by Nishi’s paper[28], he uses a

1In the diagram, the composition of raws, mixes and products in not represented faithfullyto reality in terms of the proportion of nutrients present.

32


dynamic network flow model to formulate the pooling problem. Some of the re-marks below are, indeed, based on his work. Nevertheless, a considerable numberof the additional bounds he proposes arise from observations about the time-frame. As such, they cannot be transposed to the static model presented in thisproject, although they may still be a valuable source of insight.

Let us start from some general observations. The objective is to minimise thetotal production cost, expressed as the sum of the costs for each product. Morespecifically, we consider the amount required in each case, the fraction of rawmaterials in the mixes used, and the price of the latter. Both the price and thecomposition of the mixes are unknown, causing the problem to be nonlinear.Note that the majority of the strategies presented are the result of a continu-ous effort to improve the objective value and the bounds obtained while runningtests. Here, an account of such work is provided; I present the valid inequalitiesformulated, with a particular emphasis on the concepts and ideas at their core,and comment on the solution obtained at various steps.

4.2.1 Program formulation

First, the SDP model in (3.9) was implemented in MATLAB. The data for Ex-ample1 (from the solutions obtained by Grothey et al.[16] using SLP) were usedto set up the values of all parameters and variables, in order to test the programproduced. Furthermore, the Xk’s and Ykl’s were fixed at their

Xk =[λ:,k cm

]> [λ:,k cm

]and Ykl =

[λ:,k m:,l

]> [λ:,k m:,l

]values respectively, with λjk, mjl and cmj taken from the known solution.Indeed, for the given values of the variables, the SDP returns exactly the objectivevalue obtained by the SLP method. In fact, as the Xk, Ykl matrices are formedaccording to equations (3.4) and (3.6), not only constraints (3.9j) and (3.9k) aresatisfied, but the matrices are rank 1 by construction. Hence, the solution is,effectively, determined analytically by the solver.Although such a procedure does not really investigate the results of an SDP re-laxation of the problem yet, it allows to check that the code is indeed performingthe right operations, and confirms the mathematical validity of the model.

4.2.2 First round of tests

Next, tests were run on the three problems derived from [1], without the addi-tion of any valid inequality (this corresponds to Step 2 of the algorithm proposedin 3.3). In all cases, the relaxation returns a zero objective value. The reasonfor this can be traced back to the Xk, Ykl matrices, which were introduced inthe SDP relaxation. They are the key tools of the relaxation procedure and, assuch, they are also the source of insufficient tightness of the feasible region. Moreprecisely, the model is relaxing the equalities (3.4) and (3.6), according to the

33


process illustrated in Section 3.2. Essentially, after expressing each equality astwo separate inequalities, we drop one of them 2. We hope that equality holds atthe solution, but there is scope for the optimisation to move away from equalityas the relaxation introduces freedom for the Xk, Yk,l.We can imagine this as the matrices ‘diffusing’ the constraints over all their en-tries. In fact, the components that will be used in the objective value are keptas low as possible, while feasibility is ensured by counterbalancing them with agreater change in the remaining entries.

It seems logical, then, to focus on these matrices when trying to conceive newvalid inequalities.

Zero lower bounds for the entries in (Xk)’s and Ykl’s

First of all, notice that all the entries are the result of the multiplication of non-negative quantities, and thus must be non-negative, i.e. for all k, l,

(Xk)r,s ≥ 0, for r, s = 1, . . . , 2M, (4.1)

(Ykl)r,s ≥ 0, for r, s = 1, . . . , 2M. (4.2)

4.2.3 Second round of tests

After adding the inequalities above, the results obtained, although changed, werestill very close to zero (∼ 10−10)3. Once more, this is due to the ‘diffusion’ men-tioned before. Since some of the entries were kept small by increasing others, itwould be useful to enforce a limit on the latter group.First, we look for bounds on the variables that compose them. Then, afteranalysing the way the matrices are obtained, and what they represent, theirstructure becomes apparent, as illustrated further on.

Trivial bounds for the variables

A key remark needs to be made before proceeding any further. Here we assumethat the cost of a mix is just the sum of the costs of the fractions of raws. Infact, neither extra costs nor discounts are allowed. In reality, that is not alwaysthe case. Sometimes a material is more difficult to handle in its pure form thanwhen in a blend, so it is possible that the cost of the mixes would be less than thecost of the raws needed. On the other hand, producing a mix may require addedcosts, in terms of the process itself or the storage of the mix obtained (we may, forexample, need to maintain the blend within a certain temperature range). In thiscase, the cost of the final mix would exceed that of the necessary components.

2Recall that the equality is, still, guaranteed if the matrices are rank 1. This last constraintis, nevertheless, nonconvex, hence we drop it to obtain the relaxed model.

3This really should be seen as equivalent to zero, as it is most likely the are artefact of solvertolerances. It is likely that, while initially CVX obtains a value of zero at a presolver stage,hence returning exact zero, it now requires SeDuMi to be employed in order to get, essentially,the same result.

34


For this project, such scenarios are not contemplated, and costs are derived in astraightforward manner.In this set up, we will not be able to spend less than if we were to only use thecheapest raw material (with unit price of cmin = min

ici). Similarly, costs will not

exceed those of production with only the most expensive one (cmax = maxici).

Hence, we can add the following constraint to the model:

minici ≤ cmj ≤ max

ici. (4.3)

Next, focus on the mjl, and recall it is the unit content of nutrient l in premix j.Notice that we do not know the composition of any of the mixes. Here, though,we are considering a unit of such blends, and trying to infer some informationabout the nutrients it contains.Pick any mix, and focus on one specific nutrient. No matter how we obtain themix, each unit will contain at least the same amount as a unit of the raw materialpoorest in that nutrient. Similarly, there will not be more than in a unit of theraw material that is richest in its content.Let us define the variables mmin

l and mmaxl to be, respectively, the minimum

and maximum unit content of nutrient l among all the raws. According to ourobservation, mmin

l = miniri,l and mmax

l = maxiri,l. Then, we have

mminl ≤ mjl ≤ mmax

l , ∀j. (4.4)

Restricting the (Xk)’s

Let us now consider the Xk’s more closely. Note that we can drop the index,both for the outer product matrix and for the vectors that are used to form it,i.e. we can simply work with X , λ and cm4, where

Xk :=

[λ:,kcm

] [λ:,kcm

]>→ X =

[λcm

] [λcm

]>=

[λλT λ(cm)T

cmλT cm(cm)T

]=

[X11 X12

X21 X22

]. (4.5)

The definition of Xk, as above, comes from the process illustrated in Section 3.2.As previously stated, the equality is relaxed in order to perform the SDP relax-ation, leading to the need of formulating valid inequalities to limit the acquiredfreedom of the problem.In equation (4.5), we represent the outer product explicitly, to highlight how Xcan be seen as a block matrix, each M × M block being an outer product it-self. We denote as x

(pq)rs the (r, s)-entry in the block Xpq, for r, s = 1, . . . ,M and

p, q ∈ {1, 2}. We can exploit this structure to infer some characteristics of theentries of X .Consider the upper left block X11, which consists of the outer product of λ withitself. Recall that, from constraint (2.3h), all the entries in λ need to be non-negative. Furthermore, we know λjk is defined to be the fraction of premix j in

4In fact, the following observations will not depend on any of the indices, and will, thus,hold in general.

35


product k. Clearly, all entries also need to be smaller than or equal to 1. Sinceevery entry in X11 is the product of two entries in λ, which follow the rules above,we have

0 ≤ x (11)rs ≤ 1, for r, s = 1, . . . ,M. (4.6)

A similar approach can be taken for the remaining blocks. The bottom right one,X22, is the outer product of cm with itself. In (4.3), we derived bounds for thecmj ’s, with cmin and cmax being the minimum and maximum unit cost respectively.It follows that, since each entry in X22 is a product between unit costs, it willnever be smaller that the square of cmin, nor bigger than the square of cmax.Hence,

c2min ≤ x (22)rs ≤ c2max, for r, s = 1, . . . ,M. (4.7)

Observe that, although we know the second cheapest (and the second most ex-pensive) raw material, we cannot use that information. In fact, we do not knowthe composition of mixes: we can deduce that they will not exceed the price ofthe most expensive of the raws, nor will they cost less than the cheapest, but wecannot infer any further.

The remaining two blocks, X12 and X21, are equal, since X is symmetric. Inboth cases, the outer product of λ and cm is computed. Then

λmincmin ≤ x (12), x (21) ≤ λmaxcmax.

Since λ may be zero, this is the lowest possible value for each of the entries.Furthermore, λ can, at most, equal one. Hence, the entries in X12 and X21 cannotbe greater than cmax, i.e.

0 ≤ x (12)rs ≤ cmax, (4.8)

0 ≤ x (21)rs ≤ cmax, for r, s = 1, . . . ,M. (4.9)

We can now combine (4.6), (4.7) and (4.8) to formulate the following validconstraints in terms of the Xk’s in the SDP model (3.9), for all k:

0 ≤ (Xk)r,s ≤ 1, for r, s = 1, . . . ,M, (4.10)

c2min ≤ (Xk)r,s ≤ c2max, for r, s = M, . . . , 2M, (4.11)

0 ≤ (Xk)r,s ≤ cmax, for r = 1, . . . ,M, s = M, . . . , 2M, (4.12)

0 ≤ (Xk)r,s ≤ cmax, for r = M, . . . , 2M, s = 1, . . . ,M. (4.13)

Restricting the Ykl

An analogous argument can be applied to the Yk,l’s. Here, let us keep the indices,as the resulting characteristics will depend on them. Nevertheless, let us simplifythe notation as follows:

Ykl :=

[λ:,km:,l

] [λ:,km:,l

]>→ Ykl :=

[λkml

] [λkml

]>=

[λkλ

Tk λkm

Tl

mlλTk mlm

Tl

]=

[(Y11)kl (Y12)kl(Y21)kl (Y22)kl

]. (4.14)

36


Most of what follows holds for all k, l, hence for simplicity the indices will bedropped when unnecessary.Let us start from the upper left corner, Y11. This is, just as X11, the outer productof λ’s, so that the same reasoning as for (4.6) leads us to

0 ≤ y (11)rs ≤ 1, for r, s = 1, . . . ,M. (4.15)

Clearly, the above constraints does not depend on either k or l.

Let us now proceed to Y22, the outer product of m:,l with itself, where m:,l

extracts the information about a specific nutrient from all of the mixes. Usingthe bounds on mi,l from (4.4), it follows that the entries of the outer product ofm:,l with itself will need to fall within the square of such limits, i.e.

(mminl )2 ≤ mplmql ≤ (mmax

l )2 for p, q = 1, . . . ,M.

It follows that, for all k, l,

(mminl )2 ≤ (y

(22)kl )rs ≤ (mmax

l )2, for r, s = 1, . . . ,M. (4.16)

We are left with Y12 and Y21. As for the X ’s, they are equal because of thesymmetry of the Y ’s. In both cases, they consist of the outer product of λ andm:,l, such that

λminmminl ≤ (y

(12)kl ), (y

(21)kl ) ≤ λmaxm

maxl .

As before, notice λ may be zero, so that this will be the lower limit for the entries.Furthermore, using the fact that λ can, at most, equal one, and the entries in(Y12)kl and (Y21)kl cannot be greater than mmax

l , we get for all k, l:

0 ≤ (y(12)kl ) ≤ mmax

l , (4.17)

0 ≤ (y(21)kl ) ≤ mmax

l , for r, s = 1, . . . ,M. (4.18)

We can formulate (4.15), (4.16) and (4.17) in terms of the Ykl’s in the SDPmodel (3.9) to get

0 ≤ (Ykl)r,s ≤ 1, for r, s = 1, . . . ,M, (4.19)

(mminl )2 ≤ (Ykl)r,s ≤ (mmax

l )2, for r, s = M, . . . , 2M, (4.20)

0 ≤ (Ykl)r,s ≤ mmaxl , for r = 1, . . . ,M, s = M, . . . , 2M, (4.21)

0 ≤ (Ykl)r,s ≤ mmaxl , for r = M, . . . , 2M, s = 1, . . . ,M, (4.22)

for all k, l.

37


4.2.4 Third round of tests

When tests were rerun, although a change was detected within the matrices, thefinal objective value remained very close to zero (∼ 10−8)5. A final set of con-straints stem naturally from an observation about the cost of the mixes.

Bounding the objective value

Let us take a step back and consider the general picture. We know the demand foreach of the products, from which it is trivial to obtain the total demand. Usingthis information, we can already deduce bounds on the total cost of production,since we know the prices for all the available raws.Using the bounds on the cmj ’s from (4.3), we can formulate the above reasoningas

cmin

D∑k=1

tk ≤D∑k=1

tk

M∑j=1

λjkcmj ≤ cmax

D∑k=1

tk.

Obviously, we want to express this inequality in the form of the SDP relaxation,as we should be careful not to introduce nonlinearity again while trying to reducethe feasible set. Hence, using A as defined in (3.5), we have

cmin

D∑k=1

tk ≤D∑k=1

tk Tr(XkA) ≤ cmax

D∑k=1

tk, (4.23)

or, equivalently,

D∑k=1

tk Tr(XkA)− cminD∑k=1

tk ≥ 0, (4.24)

cmax

D∑k=1

tk −D∑k=1

tk Tr(XkA) ≥ 0, (4.25)

which can be added to (3.9).

4.2.5 Results and further ideas

Finally, the objective value improved considerably, as shown in the table below.Note that the table shows the results for Grothey1 too, which was constructedduring the final stages of this project to run more tests.

A few remarks need to be made. First of all, although the objective value is,indeed, greater than zero, the SDP essentially settled on the lowest possible valuesatisfying (4.32c) and (4.32d). Hence, little or no benefit is gained by perform-ing the SDP relaxation, as those constraints themselves could provide the same

5See footnote 3.

38


SLP [16] SDP (Step 2) SDP & V.I.(Step 2bis)

Example1206.25000245.00000269.66667355.30952

0 150

Example3 133.58696 0 80

Foulds2580059006000

0 1800

Grothey1479.81083545.13891798.92180851.94832

0 225

Table 4.2: Results from sequential linear programming (SLP), the proposed SDPrelaxation, and the SDP relaxation with valid inequalities (V.I.).

bounds. Many algorithms rely on bounds to search for solutions (e.g. the Branchand Bound named in Section 2.6), with their efficacy and efficiency depending onthe quality of the bounds used. Thus, tightness is a key aspect to be consideredwhen developing relaxation methods.If we compare the minima obtained by Grothey et al. [16] using sequential linearprogramming (SLP) with the bounds provided by the SDP solutions (with theadditional valid inequalities), we notice there is often a considerable differencebetween these values, suggesting the quality of the bounds may be poor. Moretests should be run using examples of different sizes, before being able to draw adefinitive conclusion.

The following ideas were also considered, but were not implemented in thecomputational model.

Working on second order monomials

To obtain some more valid inequalities, we can take the approach suggested byNishi[28] for monomials of degree 2. Consider a monomial xy, with variablesbounded above and below such that 0 ≤ Lx ≤ x ≤ Ux and 0 ≤ Ly ≤ y ≤ Uy.Then

xy ≤ Uyx+ Lxy − LxUy, xy ≤ Lyx+ Uxy − UxLy, (4.26)

xy ≥ Uyx+ Uxy − UxUy, xy ≥ Lyx+ Lxy − LxLy. (4.27)

Note that the lower bounds obtained earlier for λ:,k(cm)T and λ:,km

T:,l can

39


be deduced from the above inequalities, with most terms cancelling out due toLλ = 0. We may want to add more constraints to the ones obtained so far forthe entries in the Xk’s and Ykl’s. Nevertheless, for the other outer products wealready have bounds that are constant, and can be easily obtained before evensolving the SDP relaxation. On the other hand, the inequalities above need aconsiderable number of computations and extra constraints, and the benefits wemay gain from adding them may not justify their use.

Consider now the case where x = y. Then, inequalities (4.26) become

x2 ≤ (Ux + Lx)x− LxUx, (4.28)

x2 ≥ 2Uxx− U2x , x2 ≥ 2Lxx− L2

x. (4.29)

It is possible to use such limits for all the unknown variables. We alreadyhave, though, lower and upper limits for the µij’s, from constraints (2.3d)and(2.3e), and the λjk’s, from (2.3g) and (2.3h) (we used these last ones earlier inthe section). Furthermore, bounds were deduced for the mjl in (4.4) and the cmjin (4.3).

In this case, as before, it is necessary to reason on the benefits of addingsuch new constraints. These changes would affect the dimensions of the prob-lem and, as a consequence, the computational cost of solving it. Our doubts areconfirmed by Nishi [28], who documents some tests run adding different sets ofvalid constraints to his model and compares the characteristics of the problemin each case. The introduction of the inequalities derived from the properties ofmonomials of degree 2 increase considerably the size of the problem, if comparedto the relaxation with the addition of some valid bounds. In most cases, thenumber of variables required doubles with respect to the primal problem, whilethere are on average more than six times as many constraints. As expected,computational time is also affected, especially for the larger problems (while,at best, it approximately doubles, it takes more than twenty times as long toreach a solution in the worst case). In many instances, the improvements in theobjective value are not substantial, despite the additional calculations performed.

In view of the intuitions above, and the results from [28] in their support, itis reasonable to discard the idea of adding such inequalities in each and everycase. Nevertheless, one may want to use some of them for a particular instance.In fact, it may be possible to attribute the source of excessive relaxation of thefeasible region to some specific variables. Then, adding the corresponding extraconstraints may well be worth the computational repercussions.

The concept of valid inequalities is worth exploring more, as it is open to ex-perimentation from various perspectives. The next section presents some sugges-tions for further work and implementations, while an overview of the constraintsto be added to the algorithm in 3.3 is provided at the end of the chapter.

40


4.3 Is there scope for improvement?

There are several ideas and developments that could be the focus of furtherresearch, which may answer some questions left open and tackle some unresolvedissues. They can be divided in three areas, each covering an essential aspect ofthis project: the computational implementation, the tests and their analysis, andthe mathematical model.

4.3.1 Writing a versatile code

When modelling a problem, it is essential to have a general algorithm that canbe, if necessary, modified for specific cases. This is particularly true for com-puter programming: it is crucial to produce a code that can adapt to problemsof variable sizes, and handle exceptions in a satisfactory manner. In reality, onesometimes starts from a particular instance, that may be easier to approach, tothen generalise the code produced.

For the computational work in this project, MATLAB was the environmentof choice, as it is one of the most used languages for numerical computation andprogramming in mathematics. In order to solve an SDP program, it is neces-sary to use an external tool. CVX seemed a suitable choice, as it allows to modelproblems in a MATLAB-based environment, and has built-in commands to utiliseexternal solvers (as mentioned before, SeDuMi and SDP3 were used).

The model proposed in this project has a specific structure. In fact, whenthe SPD relaxation is performed, some matrices Xk’s and Ykl’s are created, asexplained in Section 3.2. Each of these needs to satisfy some constraints, such as(3.9i)-(3.9k), which depend on the indices k and l. The same applies for most ofthe valid inequalities suggested in the previous sections. To be able to performthe computations efficiently, and to gain a better understanding of the effects ofthe inequalities formulated, it is essential to be able to loop over the Xk’s andYkl’s.In MATLAB, it is possible to use a cell array. This is a way to store a set ofdata, each with the same type and structure, in what is essentially an indexedmultidimensional array of cells. This would allow us to loop over the indices toenforce the constraints, while making sure to preserve the symmetry of the matri-ces used6. Unfortunately, CVX does not accept a cell array as a primal variable.As a consequence, every constraint had to be explicitly written out for all thedifferent indices, rendering the code very inefficient. This has seriously limitedthe range of problems that could serve as examples, particularly in terms of theirdimensions (large problems would prove to be very cumbersome and prone totypos and coding errors).

To work around the problem, I considered using a multidimensional array (sothat we would have a single matrix X ∈ R2M×2M×D and a Y ∈ R2M×2M×D×L), as

6Recall that the Xk’s and Ykl’s are the result of outer products, and thus symmetric.

41


well as two larger 2-dimensional matrices (X ∈ R2MD×2M and Y ∈ R2MD×2MN).A new problem arises in both cases: the submatrices that comprise these largerstructures require symmetry imposed upon them, but not upon the structure theyform, i.e. we want a non-symmetric structure containing symmetric matrices.A possible approach would be to enforce symmetry indirectly. For instance, wecould add some constraints to guarantee that each submatrix equals its transpose,hence making it symmetric. First of all, this is computationally inefficient. Infact, when storing a symmetric matrix, its structure is exploited by the programso to only store half of the entries, saving memory space. Furthermore, althoughthis trick works mathematically, the software does not recognise the implicit sym-metry defined, and encounters problems when performing the optimisation.

The lack of abstractness of the code has affected the possibility of runningtests. A variety of solvers is available on the internet, and future work shouldfocus on finding an environment that can comply with structural requirements ofthe algorithm proposed.

4.3.2 Running additional tests and analysis

Once solved this programming issue, there is scope for a variety of further im-provements.First, it would be essential to test the proposed algorithm on more problems,including some larger examples and, eventually, some real-sized ones.It would, then, be possible to experiment more with the idea of adding inequali-ties, comparing the results obtained and the improvements achieved for differentsets of data and initial set-ups. Furthermore, it would be interesting to considerthe computational efficiency of the algorithm, especially for larger problems. Inorder to do this, one could take a number of tests, possibly of a variety of dimen-sions, and solve them

• with the SLP program from [16],

• with a LP relaxation approach (i.e. ignore the mixes, and blend the rawsdirectly to obtain the products), and

• with the algorithm laid out in 3.3, using the valid inequalities proposed in4.2.

One could then perform a more detailed analysis, considering both the compu-tational time needed and the solution obtained in each case. Recall that SLPsearches for an optimal solution to the original problem (2.3), although this maynot be the global one. On the other hand, the remaining two methods only pro-vide some bounds on the objective value.Given all these results and statistics, it would be possible to gain a more rigor-ous appreciation of the tightness of the bounds obtained, and better assess thebenefits of the procedure considered.

42


4.3.3 Considering more valid inequalities

The potential of adding valid inequalities to reinforce bounds and tighten thefeasible region can, also, be explored further, and a variety of approaches couldbe taken in this regard. One may focus on the graphical interpretation of thepooling problem mentioned in Chapter 2 [16][17], as some observations about thestructure of the model could give rise to new inequalities.

In addition, the inequalities suggested for monomials of degree 2 could bereconsidered. Having access to a broader range of results and tests, one may tryto apply only some of those constraints, as suggested before, to bound specificvariables. Tests results could then help in deciding whether such additions arejustifiable.

One final idea was noticed, which could not be tested because of lack oftime and issues with the solver. The final products are, in the pooling problem,obtained after mixing raws in intermediate pooling tanks. In our model the prop-erties of such mixes are inherited directly from the raws, without any alterations.Hence, the same products could be obtained by mixing the raws directly. Asmentioned in Chapter 2, the case when there are no mixing bins is a blendingproblem, which is linear. Maintain the notation used for the pooling program in(2.3), and let wik be the (unknown) fraction of raw material i in product k, thenthe blending problem is as follows:

minD∑k=1

tk

R∑i=1

wikci (4.30a)

s.t. dlk ≤R∑i=1

wikril ≤ dlk, ∀k, l (4.30b)

R∑i=1

wik = 1, ∀k (4.30c)

wik ≥ 0, ∀i, j. (4.30d)

One strategy involves solving the problem above, which is a simple linear pro-gram, to obtain a lower bound for the original pooling problem. In fact, (4.30)is essentially just an LP relaxation of (2.3). Even imposing such a bound onthe objective value before solving the SDP relaxation, the solution returned bythe latter may still not improve. In fact, we observed earlier that the entries inthe Xk, Yk,l matrices tend to ‘diffuse’ the impact of additional constraints. Thatis, the entries that influence the objective value are only modified of the mini-mum amount necessary to preserve the feasibility of all entries. Meanwhile, theremaining ones are changed more substantially to satisfy the constraints, whilemaintaining the objective value as low as possible.

A potentially more promising approach aims to restrict the composition ofmixes and products. In fact, the latter not only need to respect the upper and

43


lower bounds required by the clients, but also need to be hypothetically obtainablestraight from the raw materials. That is, for all k, l, we can add the followingconstraint:

R∑i=1

w(i, k)r(i, l) = Tr(YklA), (4.31)

in order to try to restrain the behaviour of the Ykl’s.

44


4.4 Valid inequalities: summary

Below is a summary of the constraints to be added to the algorithm in 3.3.

for j = 1, 2, . . . ,M

minici ≤ cmj ≤ max

ici (4.32a)

for l = 1, 2, . . . , N

mminl ≤ mjl ≤ mmax

l (4.32b)

end for

end for

D∑k=1

tk Tr(XkA)− cminD∑k=1

tk ≥ 0 (4.32c)

cmax

D∑k=1

tk −D∑k=1

tk Tr(XkA) ≥ 0 (4.32d)

for k = 1, 2, . . . , D

for r = 1, 2, . . . , 2M

for s = 1, 2, . . . , 2M

(Xk)r,s ≥ 0 (4.33a)

end for

end for

for r = 1, 2, . . . ,M

for s = 1, 2, . . . ,M

(Xk)r,s ≤ 1 (4.33b)

end for

end for

for r = M, . . . , 2M

for s = M, . . . , 2M

c2min ≤ (Xk)r,s ≤ c2max (4.33c)

end for

end for

for r = 1, 2, . . . , 2M

for s = M, . . . , 2M

0 ≤ (Xk)r,s ≤ cmax (4.33d)

end for

end for

for r = M, . . . , 2M

for s = 1, 2, . . . , 2M

0 ≤ (Xk)r,s ≤ cmax (4.33e)

end for

end for

end for

for k = 1, 2, . . . , D

for l = 1, 2, . . . , N

for r = 1, 2, . . . , 2M

for s = 1, 2, . . . , 2M

(Ykl)r,s ≥ 0 (4.34a)

end for

end for

for r = 1, 2, . . . , 2M

for s = 1, 2, . . . , 2M

(Ykl)r,s ≤ 1 (4.34b)

end for

end for

for r = M, . . . , 2M

for s = M, . . . , 2M

(mminl )2 ≤ (Ykl)r,s ≤ (mmax

l )2 (4.34c)

end for

end for

for r = 1, 2, . . . , 2M

for s = M, . . . , 2M

0 ≤ (Ykl)r,s ≤ mmaxl (4.34d)

end for

end for

for r = M, . . . , 2M

for s = 1, 2, . . . , 2M

0 ≤ (Ykl)r,s ≤ mmaxl (4.34e)

end for

end for

end for

end for

45

Conclusions

This projects aims to apply a semidefinite programming relaxation to the poolingproblem.I first present an overview of the two main areas interested, namely semidefiniteprogramming and the pooling problem. I treat a selected number of topics thatconstitute the core of the subjects, with a specific focus on the key notions thatare later employed to develop the model proposed. For some important areas thatare only marginal to the work presented, an overview is given for completeness,together with suggestions for further readings.

Semidefinite programming is an extension of linear programming over the coneof positive semidefinite matrices that attracted the attention of experts from avariety of areas since the 1990s, mainly due to its application to combinatorialoptimisation and control theory. A general semidefinite programming problemformulation is given, together with the main points of duality theory for the field.Interior point methods are the most common algorithm used to solve SDP, andthe progress in their efficiency further increased the popularity of SDP. A briefoverview of the theory is provided, as they are employed by the optimisationsolvers used for the computational work in this project. Furthermore, a semidef-inite programming relaxation to combinatorial optimisation is presented. Thechoice to focus on this application is due to similarity of such approach to thetechniques used for the pooling problem in later chapters.

The pooling problem is a common issue in the petroleum industry, althoughit can arise in variety of other fields. In summary, different types of crude oils aredistilled to produce a number of intermediate mixers, which will then be use toblend the desired final products; these must meet quality requirements set by thecostumers, normally expressed as value per unit amount of some components.The presence of intermediate tanks introduces nonlinearity through bilinear con-straints, making the problem difficult to solve. Different models can be used toformulate the problem, with the bilinear being the one used for this project. Botha conceptual and mathematical derivation of our model are presented, togetherwith an overview of the formulations available in the literature, which differ forthe use of total flow variables or proportional ones. Finally, I briefly discuss theimplications of nonconvexity and nonlinearity, that often make computing theglobal optimum unrealistic and motivate methods to obtain a good enough solu-tion (generally not global) in an efficient manner.

46


The second half of the report summarises the work implemented. First, I pro-vide an overview of the work that inspired this project, in particular the effortspresented in [1], [23] and [28]. I then propose an algorithm to apply a semidefiniteprogramming relaxation to the pooling problem, similar to the one showed forthe combinatorial optimisation. The idea is to reformulate the bilinear term inthe model, in order to obtain a lower bound for the solution. Such an approachcauses an excessive relaxation of the feasible region, leading to poor bounds. Ithen elaborate on the concept of valid constraints suggested in [28], in order totighten the feasible region and get better results.

The ideas developed were implemented computationally using MATLAB, andsome numerical tests were run. The final chapter looks in details into the im-plementation of a computational version of the model, and guides the readerthrough the derivation of valid inequalities, which were conceived in an attemptto reduce the difference between the bound provided by the SDP relaxation andthe best known local optimum of the problem studied. In each case, the resultsobtained are compared to the known optima, often showing a considerable differ-ence between these values. Although this suggests the quality of the bounds maybe poor, further computational testing is required, especially using examples ofdifferent sizes, before being able to draw a definitive conclusion.

Finally, some specific suggestions for further work and implementations arepresented. They cover three main areas at the core of the project, namely thecomputational implementation, the tests and their analysis, and the mathemat-ical model. In each of these areas, there is scope for improvement, and furtherresearch may answer some questions left open and tackle some unresolved issues.

47

Bibliography

[1] N. Adhya, M. Tawarmalani, and N. V. Sahinidis. A lagrangian approachto the pooling problem. Industrial and Engineering Chemistry Research,38:1956–1972, 1999.

[2] F. Alizadeh. ombinatorial Optimization with Interior Point Methods andSemidefinite Matrices. PhD thesis, University of Minnesota, 1991.

[3] F. Alizadeh. Interior point methods in semidefinite programming with ap-plications to combinatorial optimization. SIAM Journal on Optimization,5:13–51, 1993.

[4] C. Audet, J. Brimberg, P. Hansen, S. Le Digabel, and N. Mladenovic. Pool-ing Problem: Alternate Formulations and Solution Methods. ManagementScience, 50(6):761–776, June 2004.

[5] J. Bisschop. Optimization Modelling, chapter 12, pages 123–132. ParagonDecision Technology B.V., 2006.

[6] S. Boyd and L. Vandenberghe. Semidefinite programming relaxations of non-convex problems in control and combinatorial optimization. In A. Paulraj,V. Roychowdhury, and C. Schaper, editors, Communications, Computation,Control, and Signal Processing, pages 279–287. Springer US, 1997.

[7] S. Boyd and L. Vandenberghe. Convex Optimization. Berichte uber verteiltemessysteme. Cambridge University Press, 2004.

[8] I. CVX Research. CVX: Matlab software for disciplined convex program-ming, version 2.0. http://cvxr.com/cvx, Aug. 2012.

[9] G. B. Dantzig. The diet problem. Interfaces, 20(4):43–47, Jul. - Aug. 1990.

[10] E. De Klerk. Interior Point Methods for Semidefinite Programming. 1997.

[11] C. A. Floudas and A. Aggarwal. A decomposition strategy for global opti-mum search in the pooling problem. ORSA Journal on Computing, 2(3):225–235, summer 1990.

[12] L. Foulds, D. Haugland, K. Jornsten, and C. M. I. C. for Petroleum Eco-nomics. A Bilinear Approach to the Pooling Problem. Rapport (Chr.Michelsens institutt. Avdeling for naturvitenskap og teknologi : trykt utg.).Chr. Michelsen Institute, Centre for Petroleum Economics, 1991.

48

http://cvxr.com/cvx


[13] R. M. Freud. Introduction to semidefinite programming. MassachusettsInstitute of Technology, Cambridge, MA, May 1998.

[14] J. Gondzio. Large scale optimization lectures.

[15] J. Gondzio. Interior point methods 25 years later. European Journal ofOperational Research, 218(3):587–601, 2012.

[16] A. Grothey, R. Fawcett, and K. McKinnon. Solving a feed-mix poolingproblem in agriculture. In OR41 Conference Proceedings, 1999.

[17] D. Hamilton. Decomposition and Diet Problem. PhD thesis, University ofEdinburgh, 2009.

[18] C. A. Haverly. Studies of the behavior of recursion for the pooling problem.SIGMAP Bull., (25):19–28, Dec. 1978.

[19] C. Helmberg. Semidefinite programming. European Journal of OperationalResearch, 137(3):461 – 482, 2002.

[20] F. S. Hillier and G. J. Lieberman. Introduction to Operations Research.McGraw-Hill, 7th edition, 2005.

[21] H. Hindi. A tutorial on convex optimization ii: duality and interior pointmethods, 2006.

[22] N. Karmarkar. A new polynomial-time algorithm for linear programming,1984.

[23] J. Lavaei and S. Low. Zero duality gap in optimal power flow problem. PowerSystems, IEEE Transactions on, 27(1):92 –107, February 2012.

[24] G. McCormick. Computability of global solutions to factorable nonconvexprograms: Part i convex underestimating problems. Mathematical Program-ming, 10:147–175, 1976.

[25] R. Misener and C. Floudas. Advances for the pooling problem: Modeling,global optimization, and computational studies. Applied and ComputationalMathematics, 8(1):3–22, 2009.

[26] A. Nemirovski. Lecture notes: Interior point polynomial time methods inconvex programming. Spring Semester 2004.

[27] Y. Nesterov and A. S. Nemirovskii. Interior-point polynomial algorithms inconvex programming, volume 13. Society for Industrial and Applied Mathe-matics, 1994.

[28] T. Nishi. A Semidefinite Programming Relaxation Approach for the PoolingProblem. PhD thesis, Kyoto University, Department of Applied Mathematicsand Physics, 2010.

49


[29] F. A. Potra and S. J. Wright. Interior-point methods. In Society for Industrialand Applied Mathematics (SIAM. SIAM, 2000.

[30] R. Saigal, H. Wolkowicz, and L. Vandenberghe. Handbook of SemidefiniteProgramming: Theory, Algorithms, and Applications. International Series inOperations Research and Management Science Series. Kluwer, 2000.

[31] L. Vanderberghe and S. Boyd. Semidefinite programming. SIAM Review,38(1):49–95, March 1996.

[32] S. Wright. Primal-Dual Interior-Point Methods. Miscellaneous Bks. Societyfor Industrial and Applied Mathematics, 1997.

50

Documents

Can semidefinite programming be a key approach to the ... · Semide nite programming (or SDP) is an extension of linear programming over the cone of positive semide nite matrices