An Integer Programming Approach forLinear Programs with Probabilistic Constraints

James Luedtke, Shabbir Ahmed, and George Nemhauser

H. Milton Stewart School of Industrial and Systems EngineeringGeorgia Institute of Technology

Atlanta, GA, USA{jluedtke,sahmed,gnemhaus}@isye.gatech.edu

Abstract. Linear programs with joint probabilistic constraints (PCLP)are known to be highly intractable due to the non-convexity of the feasi-ble region. We consider a special case of PCLP in which only the right-hand side is random and this random vector has a finite distribution.We present a mixed integer programming formulation and study the re-laxation corresponding to a single row of the probabilistic constraint,yielding two strengthened formulations. As a byproduct of this analy-sis, we obtain new results for the previously studied mixing set, subjectto an additional knapsack inequality. We present computational resultsthat indicate that by using our strengthened formulations, large scaleinstances can be solved to optimality.

Keywords: Integer programming, probabilistic constraints, stochasticprogramming.

1 Introduction

Consider a linear program with a probabilistic or chance constraint

(PCLP ) min{cx : x ∈ X, P{Tx ≥ ξ} ≥ 1 − ε

}(1)

where X ={x ∈ R

d+ : Ax = b

}is a polyhedron, c ∈ R

d, T is an m × d randommatrix, ξ is a random vector taking values in R

m, and ε is a confidence parameterchosen by the decision maker, typically near zero, e.g., ε = 0.01 or ε = 0.05. Notethat in (1) we enforce a single probabilistic constraint over all rows, rather thanrequiring that each row independently be satisfied with high probability. Sucha constraint is known as a joint probabilistic constraint, and is appropriate in acontext in which it is important to have all constraints satisfied simultaneouslyand there may be dependence between random variables in different rows.

Problems with joint probabilistic constraints have been extensively studied;see [1] for background and an extensive list of references. Probabilistic constraintshave been used in various applications including supply chain management [2],production planning [3], optimization of chemical processes [4,5] and surfacewater quality management [6]. Unfortunately, linear programs with probabilistic

M. Fischetti and D.P. Williamson (Eds.): IPCO 2007, LNCS 4513, pp. 410–423, 2007.c© Springer-Verlag Berlin Heidelberg 2007

Integer Programming Approach for Probabilistic Constraints 411

constraints are still largely intractable except for a few very special cases. Thereare two primary reasons for this intractability. First, in general, for a givenx ∈ X , the quantity φ(x) := P{Tx ≥ ξ} is hard to compute, as it requires multi-dimensional integration. Second, the feasible region defined by a probabilisticconstraint is generally not convex.

Recently, several approaches have been proposed which can find highly reli-able feasible solutions to probabilistic programs. Examples of these conservativeapproximations include scenario approximation [7,8], Bernstein approximation[9] and robust optimization, e.g., [10,11,12]. These methods are attractive whenhigh reliability is most important and solution cost is a secondary objective.However, when very high reliability is not crucial, for example if the proba-bilistic constraint represents a service level constraint, a decision maker may beinterested in exploring the trade-off between solution cost and system reliability,and would be interested in obtaining solutions which are on or near the efficientfrontier of these competing objectives. The aforementioned conservative approx-imations generally do not yield bounds on the optimal solution cost at a givenreliability level ε, and hence cannot distinguish whether the produced solutionsare close to the efficient frontier. This latter context is the motivation for usinginteger programming to solve PCLP so that we can obtain solutions that areprovably optimal or near optimal.

In this work, we demonstrate that by using integer programming techniques,PCLP can be solved efficiently under the following two simplifying assumptions:

(A1) Only the right-hand side vector ξ is random; the matrix T = T is deter-ministic.

(A2) The random vector ξ has a finite distribution.

Despite its restrictiveness, the special case given by assumption A1 has received alot of attention in the literature, see, e.g., [1,13,14]. A notable result for this caseis that if the distribution of the right-hand side is log-concave, then the feasibleregion defined by the joint probabilistic constraint is convex [15]. This allowsproblems with small dimension of the random vector to be solved to optimal-ity, but higher dimensional problems are still intractable due to the previouslymentioned difficulty in checking feasibility of the probabilistic constraint. Spe-cialized methods have been developed in [14] for the case in which assumptionA1 holds and the random vector has discrete but not necessarily finite distri-bution. However, these methods also do not scale well with the dimension ofthe random vector. Assumption A2 may also seem very restrictive. However, ifthe possible values for ξ are generated by taking Monte Carlo samples from ageneral distribution, we can think of the resulting problem as an approxima-tion of the problem with this distribution. Under some reasonable assumptionswe can show that the optimal solution of the sampled problem converges expo-nentially fast to the optimal solution of the original problem as the number ofscenarios increases. Also, the optimal objective of the sampled problem can beused to develop statistical lower bounds on the optimal objective of the originalproblem. See [16,17,18] for some related results. It seems that the reason sucha sampling approach has not been seriously considered for PCLP in the past is

412 J. Luedtke, S. Ahmed, and G. Nemhauser

that the resulting sampled problem has a non-convex feasible region, and thusis still generally intractable. Our contribution is to demonstrate that, at leastunder assumption A1, it is nonetheless possible to solve the sampled problem inpractice.

Under assumption A2 it is possible to write a mixed integer programmingformulation for PCLP, as has been done, for example, in [19]. In the generalcase, such a formulation requires the introduction of “big-M” type constraints,and hence is difficult to solve. However, the particular case of assumption A1 hasnot been studied from an integer programming perspective; by doing so, we areable to develop strong mixed integer programming formulations. Our approachin developing these formulations is to consider the relaxation obtained from asingle row in the probabilistic constraint. It turns out that this yields a systemsimilar to the mixing set introduced by Gunluk and Pochet [20], subject to anadditional knapsack inequality. We are able to derive strong valid inequalities forthis system by first using the knapsack inequality to “pre-process” the mixingset, then applying the mixing inequalities of [20], see also [21,22]. We also derivean extended formulation, equivalent to one given by Miller and Wolsey in [23].Making further use of the knapsack inequality, we are able to derive more generalclasses of valid inequalities, for both the original and extended formulations. Ifall scenarios are equally likely, the knapsack inequality reduces to a cardinalityrestriction. In this case, we are able to characterize the convex hull of feasiblesolutions to the extended formulation for the single row case. Although theseresults are motivated by the application to PCLP, they can be used in anyproblem in which a mixing set appears along with a knapsack constraint.

2 The MIP Formulation

We now consider a probabilistically constrained linear programming problem,with random right-hand side given by

(PCLPR) min cxs.t. Ax = b

P{Tx ≥ ξ} ≥ 1 − εx ≥ 0 .

(2)

Here A is an r × d matrix, b ∈ Rr, T is an m × d matrix, ξ is a random vector

in Rm, ε ∈ (0, 1) (typically small) and c ∈ R

d. We assume that ξ has finitesupport, that is there exist vectors, ξi, i = 1, . . . , n such that P{ξ = ξi} = πi foreach i where πi ≥ 0 and

∑ni=1 πi = 1. We will refer to the possible outcomes as

scenarios. We assume without loss of generality that ξi ≥ 0 and πi ≤ ε for eachi. We also define the set N = {1, . . . , n}.

Before proceeding, we note that PCLPR is NP-hard even under assumptionsA1 and A2.

Theorem 1. PCLPR is NP-hard, even in the special case in which πi = 1/nfor all i ∈ N , the constraints Ax = b are not present, T is the m × m identitymatrix, and c = (1, . . . , 1) ∈ R

m.

Integer Programming Approach for Probabilistic Constraints 413

We now formulate PCLPR as a mixed integer program [19]. To do so, we in-troduce for each i ∈ N , a binary variable zi, where zi = 0 will guarantee thatTx ≥ ξi. Observe that because ε < 1 we must have Tx ≥ ξi for at least onei ∈ N , and because ξi ≥ 0 for all i, this implies Tx ≥ 0 in any feasible solutionof PCLPR. Then, introducing variables v ∈ R

m to summarize Tx, we obtain theMIP formulation of PCLPR given by

(PMIP ) min cx

s.t. Ax = b, Tx − v = 0 (3)v + ξizi ≥ ξi i = 1, . . . , n (4)

n∑i=1

πizi ≤ ε (5)

x ≥ 0, z ∈ {0, 1}n.

3 Strengthening the Formulation

Our approach is to strengthen PMIP by ignoring (3) and finding strong formu-lations for the set

F :={(v, z) ∈ R

m+ × {0, 1}n : (4), (5)

}. (6)

Note that

F =m⋂

j=1

{(v, z) : (vj , z) ∈ Gj} ,

where for j = 1, . . . , m

Gj = {(vj , z) ∈ R+ × {0, 1}n : (5), vj + ξijzi ≥ ξij i = 1, . . . , n} .

Thus, a natural first step in developing a strong formulation for F is to develop astrong formulation for each Gj . In particular, note that if an inequality is facet-defining for conv(Gj), then it is also facet-defining for conv(F ). This followsbecause if an inequality valid for Gj is supported by n + 1 affinely independentpoints in R

n+1, then because this inequality will not have coefficients on vi forany i �= j, the set of supporting points can trivially be extended to a set of n+maffinely independent supporting points in R

n+m by appropriately setting the vi

values for each i �= j.The above discussion leads us to consider the generic set

G = {(y, z) ∈ R+ × {0, 1}n : (5), y + hizi ≥ hi i = 1, . . . , n} (7)

obtained by dropping the index j in the set Gj and setting y = vj and hi = ξij

for each i. For convenience, we assume that the hi are ordered so that h1 ≥ h2 ≥· · · ≥ hn. The mixing set

P = {(y, z) ∈ R+ × {0, 1}n : y + hizi ≥ hi i = 1, . . . , n}

414 J. Luedtke, S. Ahmed, and G. Nemhauser

has been extensively studied, in varying degrees of generality, by Atamturk et. al[21], Gunluk and Pochet [20], Guan et. al [22] and Miller and Wolsey [23]. If weignore the knapsack constraint in G, we can apply these results to obtain theset of valid inequalities

y +l∑

j=1

(htj − htj+1)ztj ≥ ht1 ∀T = {t1, . . . , tl} ⊆ N , (8)

where t1 < t2 < · · · < tl and htl+1 := 0. Following [21], we call (8) the starinequalities. In addition, these inequalities can be separated in polynomial time[20,21,22]. It has been shown in these same works that these inequalities definethe convex hull of P and are facet defining if and only if t1 = 1. We can doconsiderably better, however, by using the knapsack constraint in G to firststrengthen the inequalities, and then derive the star inequalities. In particular,let p := max

{k :

∑ki=1 πi ≤ ε

}. Then, due to the knapsack constraint, we cannot

have zi = 1 for all i = 1, . . . , p + 1 and thus we have y ≥ hp+1. This also impliesthat the mixed integer constraints in G are redundant for i = p+1, . . . , n. Thus,we can write a tighter formulation of G as

G = {(y, z) ∈ R+ × {0, 1}n : (5), y + (hi − hp+1)zi ≥ hi i = 1, . . . , p} . (9)

Remark 1. In addition to yielding a tighter relaxation, this description of G isalso more compact. In typical applications, ε will be near 0, suggesting p << n.When applied for each j in the set F , this will yield a formulation with mp <<mn rows.

If we now apply the star inequalities to the improved formulation of G, we obtainthe following result, which can be obtained by applying results in [20],[21] or [22].

Theorem 2. The inequalities

y +l∑

j=1

(htj − htj+1)ztj ≥ ht1 ∀T = {t1, . . . , tl} ⊆ {1, . . . , p} (10)

with t1 < . . . < tl and htl+1 := hp+1, are valid for G. Moreover, (10) is facet-defining for conv(G) if and only if ht1 = h1.

We refer to the inequalities (10) as the strengthened star inequalities.

Remark 2. The difference between the star inequalities (8) and strengthenedstar inequalities (10) is that in (10) we have htl+1 := hp+1 whereas in (8) wehave htl+1 := 0, corresponding to the fact that our lower bound on y was shiftedfrom 0 to hp+1 by using the knapsack inequality.

Remark 3. The strengthened star inequalities are not sufficient to characterizethe convex hull of G, even in the special case in which all probabilities are equal,that is πi = 1/n for all i.

Integer Programming Approach for Probabilistic Constraints 415

We now consider the special case in which πi = 1/n for all i ∈ N . Note thatin this case we have p := max

{k :

∑ki=1 1/n ≤ ε

}= �nε and the knapsack

constraint (5) becomesn∑

i=1

zi ≤ nε

which, by integrality on zi can be strengthened to the simple cardinalityrestriction

n∑i=1

zi ≤ p . (11)

Thus, the feasible region for our single row formulation becomes

G′ = {(y, z) ∈ R+ × {0, 1}n : (11), y + (hi − hp+1)zi ≥ hi i = 1, . . . , p} .

Now, observe that for any (γ, α) ∈ Rn+1, the problem

min {γy + αz : (y, z) ∈ G′}

is easy. Indeed, if γ < 0, then the problem is unbounded, so we can assumeγ ≥ 0. Then, one need only consider setting y to hk for k = 1, . . . , p + 1, andsetting the zi accordingly. That is, if y = hk for k ∈ {1, . . . , p + 1}, then we mustset zi = 1 for i = 1, . . . , k − 1. The remaining zi can be set to 0 or 1 as long as∑n

i=k zi ≤ p−k+1. Hence, we set zi = 1 if and only if i ∈ S∗k where

S∗k ∈ argmin

S⊆{k,...,n}

{∑i∈S

αi : |S| ≤ p−k+1

}.

Since we can optimize over G′ efficiently, we know that we can separate overconv(G′) efficiently. Indeed, given (y∗, z∗) we can write an explicit polynomialsize linear program for separation over conv(G′). Although this would yield atheoretically efficient way to separate over conv(G′), it still may be too expensiveto solve a linear program to generate cuts. We would therefore prefer to have anexplicit characterization of a class or classes of valid inequalities for G′ with anassociated combinatorial algorithm for separation. The following theorem givesan example of one such class.

Theorem 3. Let m ∈ {1, . . . , p − 1}, T = {t1, . . . , tl} ⊆ {1, . . . , m} and Q ={q1, . . . , qp−m} ⊆ {p+1, . . . , n} . Define Δm

1 = hm+1 − hm+2 and

Δmi = max

⎧⎨⎩Δm

i−1, hm+1 − hm+i+1 −i−1∑j=1

Δmj

⎫⎬⎭ for i = 2, . . . , p−m .

Then, with htl+1 := hm+1,

y +l∑

j=1

(htj − htj+1)ztj +p−m∑j=1

Δmj (1 − zqj ) ≥ ht1 (12)

is valid for G′ and facet-defining for conv(G′) if and only if ht1 = h1.

416 J. Luedtke, S. Ahmed, and G. Nemhauser

Example 1. Let n = 10 and ε = 0.4 so that p = 4 and suppose h1−5 ={20, 18, 14, 11, 6}. Let m = 2, T = {1, 2} and Q = {5, 6}. Then, Δ2

1 = 3 andΔ2

2 = max {3, 8 − 3} = 5 so that (12) yields

y + 2z1 + 4z3 + 3(1 − z5) + 5(1 − z6) ≥ 20 .

Separation of inequalities (12) can be accomplished by a simple modification tothe routine for separating the strengthened star inequalities. We have identifiedother classes of valid inequalities, but have so far failed to find a general classthat characterizes the convex hull of G′.

4 A Strong Extended Formulation

LetFS = {(y, z) ∈ R+ × [0, 1]n : (5), (10)} .

FS represents the polyhedral relaxation of G, augmented with the strengthenedstar inequalities. Note that the inequalities y + (hi − hp+1)zi ≥ hi are includedin FS by taking T = {i}, so that enforcing integrality in FS would yield a validsingle row formulation for the set G. Our aim is to develop a reasonably compactextended formulation which is equivalent to FS. To do so, we introduce variablesw1, . . . , wp and let

EG ={(y, z, w) ∈ R+ × {0, 1}n+p : (13) − (16)

}

where

wi − wi+1 ≥ 0 i = 1, . . . , p (13)zi − wi ≥ 0 i = 1, . . . , p (14)

y +p∑

i=1

(hi − hi+1)wi ≥ h1 (15)

n∑i=1

πizi ≤ ε . (16)

and wp+1 := 0. The variables wi can be interpreted as deciding whether or notscenario i is satisfied for the single row under consideration, and because theyare specific to this single row, the inequalities (13) can be safely added. Theinequalities (14) then ensure that if a scenario is infeasible for this row, then it isinfeasible overall, and the lower bound on y is now given by the single inequality(15). We let EF be the polyhedron obtained by relaxing integrality in EG.

Theorem 4. Proj(y,z)(EG) = G, that is, EG is a valid formulation for G.

An interesting result is that the linear relaxation of this extended formulation isas strong as having all strengthened star inequalities in the original formulation.A similar result has been proved in [23].

Integer Programming Approach for Probabilistic Constraints 417

Theorem 5. Proj(y,z)(EF ) = FS.

Because of the equivalence between EF and FS, Remark 3 holds for this formu-lation as well, that is, even in the special case in which all probabilities are equal,this formulation does not characterize the convex hull of feasible solutions of G.We therefore investigate what other valid inequalities exist for this formulation.We first introduce the notation

fk :=k∑

i=1

πi, k = 0, 1, . . . , p .

Theorem 6. Let k ∈ {1, . . . , p} and let S ⊆ {k, . . . , n} be such that∑

i∈S πi ≤ε − fk−1. Then, ∑

i∈S

πizi +∑

i∈{k...,p}\S

πiwi ≤ ε − fk−1 (17)

is valid for EG.

Now, consider the special case in which πi = 1/n for i = 1, . . . , n. Then theextended formulation becomes

EG′ ={(y, z, w) ∈ R+ × {0, 1}n+p : (11) and (13) − (15)

}.

Letting Sk = {S ⊆ {k, . . . , n} : |S| ≤ p−k+1} for k = 1, . . . , p, the inequali-ties (17) become

∑i∈S

zi +∑

i∈{k,...,p}\S

wi ≤ p−k+1 ∀S ∈ Sk, k = 1, . . . , p . (18)

Example 2. Let n = 10 and ε = 0.4 so that p = 4. Let k = 2 and S = {4, 5, 6}.Then (18) becomes

z4 + z5 + z6 + w2 + w3 ≤ 3 .

Now, let

EH ′ ={(y, z, w) ∈ R+ × [0, 1]n+p : (11), (13) − (15) and (18)

}

be the linear relaxation of the extended formulation, augmented with this set ofvalid inequalities.

Theorem 7. The convex hull of the extended formulation EG′ is given by theinequalities defining EG′ and the inequalities (18); that is, EH ′ = conv(EG′).

We close this section by noting that inequalities (18) can be separated in poly-nomial time. Indeed, suppose we wish to separate the point (z∗, w∗). Then sep-aration can be accomplished by solving

maxS∈Sk

⎧⎨⎩

∑i∈S

z∗i +∑

i∈{k,...,p}\S

w∗i

⎫⎬⎭ = max

S∈Sk

{∑i∈S

θ∗i

}+

p∑i=k

w∗i

418 J. Luedtke, S. Ahmed, and G. Nemhauser

for k = 1, . . . , p where

θ∗i ={

z∗i − w∗i i = 1, . . . , p

z∗i i = p + 1, . . . , n .

Hence, a trivial separation algorithm is to first sort the values θ∗i in non-increasing order, then for each k, find the maximizing set S ∈ Sk by search-ing this list. This yields an algorithm with complexity O(n log n + p2) = O(n2).However, by considering the values of k in the order p, . . . , 1 and updating an or-dered list of eligible indices Sk for each k, it is possible to improve the complexityto O(n log n). For the more general inequalities (17), (heuristic) separation canbe accomplished by (heuristically) solving p knapsack problems.

5 Computational Experience

We performed computational tests on a probabilistic version of the classicaltransportation problem. We have a set of suppliers S and a set of customers Dwith |D| = m. The suppliers have limited capacity Mi for i ∈ S. There is aper-unit transportation cost cij for (producing and) shipping a unit of productfrom supplier i ∈ S to customer j ∈ D. The customer demands are random andare represented by a random vector d ∈ R

m+ . We assume we must choose the

shipment quantities before the customer demands are known. We enforce thefollowing probabilistic constraint:

P{∑i∈S

xij ≥ dj , j = 1, . . . , m} ≥ 1 − ε . (19)

The objective is to minimize distribution costs subject to (19), non-negativityon the flow variables xij , and the supply capacity constraints

∑j∈D

xij ≤ Mi, ∀i ∈ S .

We randomly generated instances with the number of suppliers fixed at 40 andvarying numbers of customers and scenarios. The supply capacities and costcoefficients were generated using normal and uniform distributions respectively.For the random demands, we experimented with independent normal, dependentnormal and independent Poisson distributions. We found qualitatively similarresults in all cases, but the independent normal case yielded the most challenginginstances, so for our experiments we focus on this case. For each instance, we firstrandomly generated the mean and variance of each customer demand. We thengenerated the number n of scenarios required, independently across scenarios andacross customer locations, as Monte Carlo samples with these fixed parameters.In most instances we assumed all scenarios occur with probability 1/n, butwe also did some tests in which the scenarios have general probabilities, whichwere also randomly generated. CPLEX 9.0 was used as the MIP solver and allexperiments were done on a computer with two 2.4 Ghz processors (although

Integer Programming Approach for Probabilistic Constraints 419

no parallelism is used) and 2.0 Gigabytes of memory. We set a time limit ofone hour. For each problem size we generated 5 random instances and, unlessotherwise specified, the computational results reported are averages over the 5instances.

5.1 Comparison of Formulations

In Table 1 we compare the results obtained by solving our instances using

1. formulation PMIP given by (3) - (5),2. formulation PMIP with strengthened star inequalities (10), and3. the extended formulation of Sect. 4, but without (17) or (18).

When the strengthened star inequalities are not used, we still used the improvedformulation of G corresponding to (9). Recall that the strengthened star inequal-ities subsume the rows defining the formulation PMIP; therefore, when we us-ing these inequalities we initially add only a small subset of the mp inequalitiesin the formulation. Subsequently separating the strengthened star inequalities asneeded guarantees the formulation remains valid. For formulation PMIP withoutstrengthened star inequalities, we report the average optimality gap that remainedafter the hour time limit was reached. For the other two formulations, which werefer to as the strong formulations, we report the geometric average of the timeto solve the instances to optimality. We used ε = 0.05 and ε = 0.1, reflecting thenatural assumption that we want to meet demand with high probability.

The first observation from Table 1 is that formulation PMIP without thestrengthened star inequalities fails to solve these instances within an hour, oftenleaving large optimality gaps, whereas the instances are solved efficiently usingthe strong formulations. The number of nodes required to solve the instances forthe strong formulations is very small. The instances with equi-probable scenarioswere usually solved at the root, and even when branching was required, the rootrelaxation usually gave an exact lower bound. Branching in this case was onlyrequired to find an integer solution which achieved this bound. The instanceswith general probabilities required slightly more branching, but generally notmore than 100 nodes. Observe that the number of strengthened star inequalitiesthat were added is small relative to the number of rows in the formulation PMIPitself. For example, for ε = 0.1, m = 200 and n = 3, 000, the number of rows inPMIP would be mp = 60, 000, but on average, only 5, 541 strengthened star in-equalities were added. Next we observe that in most cases the computation timeusing the extended formulation is significantly less than the formulation withstrengthened star inequalities. Finally, we observe that the instances with gen-eral probabilities take somewhat longer to solve than those with equi-probablescenarios but can still be solved efficiently.

5.2 Testing Inequalities (18)

With small ε the root relaxation given by the extended formulation is extremelytight, so that adding the inequalities (18) is unlikely to have a positive impact on

420 J. Luedtke, S. Ahmed, and G. Nemhauser

Table 1. Average solution times for different formulations

PMIP PMIP+Star ExtendedProbabilities ε m n Gap Cuts Time(s) Time(s)Equal 0.05 100 1000 0.18% 734.8 7.7 1.1

100 2000 1.29% 1414.2 31.8 4.6200 2000 1.02% 1848.4 61.4 12.1200 3000 2.56% 2644.0 108.6 12.4

0.10 100 1000 2.19% 1553.2 34.6 12.7100 2000 4.87% 2970.2 211.3 41.1200 2000 4.48% 3854.0 268.5 662.2200 3000 5.84% 5540.8 812.7 490.4

General 0.05 100 1000 0.20% 931.8 9.0 3.9100 2000 1.04% 1806.6 55.2 13.2

0.10 100 1000 1.76% 1866.0 28.7 52.5100 2000 4.02% 3686.2 348.5 99.2

computation time. However, for larger ε, we have seen that the extended formu-lation may have a substantial optimality gap. We therefore investigated whetherusing inequalities (18) can improve solution time in this case. In Table 3 we presentresults comparing solution times and node counts with and without inequalities(18) for instances with larger ε. We performed these tests on smaller instancessince these instances are already hard for these values of ε. We observe that addinginequalities (18) at the root can decrease the root optimality gap significantly. Forthe instances that could be solved in one hour, this leads to a significant reductionin the number of nodes explored, and a moderate reduction in solution time. Forthe instances which were not solved in one hour, the remaining optimality gapwas usually, but not always, lower when the inequalities (18) were used. Theseresults indicate that when ε is somewhat larger, inequalities (18) may be helpfulon smaller instances. However, they also reinforce the difficulty of the instanceswith larger ε, since even with these inequalities, only the smallest of these smallerinstances could be solved to optimality within an hour.

5.3 The Effect of Increasing ε

The results of Table 1 indicate that the strong formulations can solve large in-stances to optimality when ε is small, which is the typical case. However, itis still an interesting question to investigate how well this approach works forlarger ε. Note first that we should expect solution times to grow with ε if onlybecause the formulation sizes grow with ε. However, we observe from Table 2that the situation is much worse than this. This table shows the root LP solvetimes and optimality gaps achieved after an hour of computation time for anexample instance with equi-probable scenarios, m = 50 rows and n = 1, 000 sce-narios at increasing levels of ε, using the formulation PMIP with strengthenedstar inequalities. Root LP solve time here refers to the time until no furtherstrengthened star inequalities could be separated. We see that the time to solve

Integer Programming Approach for Probabilistic Constraints 421

Table 2. Effects of increasing ε on an instance with m = 50 and n = 1000

ε 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90Root LP Time (s) 21.7 37.1 82.7 144.3 227.8 327.6 505.6 792.6 1142.6Optimality Gap 0.0% 0.0% 2.2% 5.8% 10.5% 16.2% 28.7% 35.1% 44.4%

the root linear programs does indeed grow with ε as expected, but the optimalitygaps achieved after an hour of computation time deteriorate even more drasti-cally with growing ε. This is explained by the increased time to solve the linearprogramming relaxations combined with a weakening of the relaxation bound asε increases.

Table 3. Results with and without inequalities (18)

Root Gap Nodes Time(s) or Gapm ε n Ext +(18) Ext +(18) Ext +(18)25 0.3 250 1.18% 0.67% 276.9 69.0 121.2 93.9

0.3 500 1.51% 0.58% 455.0 165.8 750.6 641.30.35 250 2.19% 1.50% 1259.4 409.0 563.2 408.40.35 500 2.55% 1.61% 2297.6 968.8 0.22% 0.06%

50 0.3 500 2.32% 2.00% 991.8 238.6 1.37% 1.41%0.3 1000 2.32% 1.75% 28.3 8.5 1.98% 1.66%

0.35 500 4.10% 3.31% 650.4 92.9 3.03% 2.66%0.35 1000 4.01% 3.23% 22.7 6.2 3.58% 3.17%

6 Concluding Remarks

We have presented strong integer programming formulations for linear programswith probabilistic constraints in which the right-hand side is random with finitedistribution. In the process we made use of existing results on mixing sets, andhave introduced new results for the case in which the mixing set additionallyhas a knapsack restriction. Computational results indicate that these formula-tions are extremely effective on instances in which reasonably high reliabilityis enforced, which is the typical case. However, instances in which the desiredreliability level is lower remain difficult to solve, partly due to increased size ofthe formulations, but more significantly due to the weakening of the formulationbounds. Moreover, these instances remain difficult even when using the inequal-ities which characterize the single row relaxation convex hull. This suggests thatrelaxations which consider multiple rows simultaneously need to be studied toyield valid inequalities which significantly improve the relaxation bounds forthese instances.

Future work in this area should focus on addressing the two assumptions wemade at the beginning of this paper. The finite distribution assumption can beaddressed by using the results about the statistical relationship between a prob-lem with probabilistic constraints and its Monte Carlo sample approximation

422 J. Luedtke, S. Ahmed, and G. Nemhauser

to establish methods for generating bounds on the optimal value of the originalproblem. Computational studies will need to be performed to establish the prac-ticality of this approach. We expect that relaxing the assumption that only theright-hand side is random will be more challenging. A natural first step in thisdirection will be to extend results from the generalized mixing set [23,24] to thecase in which an additional knapsack constraint is present.

Acknowledgments. This research has been supported in part by the NationalScience Foundation under grants DMI-0121495 and DMI-0522485.

References

1. Prekopa, A.: Probabilistic programming. In Ruszczynski, A., Shapiro, A., eds.:Stochastic Programming. Volume 10 of Handbooks in Operations Research andManagement Science. Elsevier (2003)

2. Lejeune, M.A., Ruszczynski, A.: An efficient trajectory method for probabilisticinventory-production-distribution problems. Operations Research (Forthcoming,2007)

3. Murr, M.R., Prekopa, A.: Solution of a product substitution problem using stochas-tic programming. In Uryasev, S.P., ed.: Probabilistic Constrained Optimization:Methodology and Applications. Kluwer Academic Publishers (2000) 252–271

4. Henrion, R., Li, P., Moller, A., Steinbach, M.C., Wendt, M., Wozny, G.: Stochasticoptimization for operating chemical processes under uncertainty. In Grotschel, M.,Krunke, S., Rambau, J., eds.: Online Optimization of Large Scale Systems. (2001)457–478

5. Henrion, R., Moller, A.: Optimization of a continuous distillation process underrandom inflow rate. Computers and Mathematics with Applications 45 (2003)247–262

6. Takyi, A.K., Lence, B.J.: Surface water quality management using a multiple-realization chance constraint method. Water Resources Research 35 (1999) 1657–1670

7. Calafiore, G., Campi, M.: Uncertain convex programs: randomized solutions andconfidence levels. Mathematical Programming, Ser. A 102 (2005) 25–46

8. Nemirovki, A., Shapiro, A.: Scenario approximation of chance constraints. Preprintavailable at www.optimization-online.org (2004)

9. Nemirovski, A., Shapiro, A.: Convex approximations of chance constrained pro-grams. Preprint available at www.optimization-online.org (2004)

10. Ben-Tal, A., Nemirovski, A.: Robust convex optimization. Mathematics of Oper-ations Research 23 (1998) 769–805

11. Bertsimas, D., Sim, M.: The price of robustness. Operations Research 52 (2004)35–53

12. Ghaoui, L.E., Lebret, H.: Robust solutions to least-squares problems with uncertaindata. SIAM Journal on Matrix Analysis and Applications 18 (1997) 1035–1064

13. Cheon, M.S., Ahmed, S., Al-Khayyal, F.: A branch-reduce-cut algorithm for theglobal optimization of probabilistically constrained linear programs. MathematicalProgramming, Ser B 108 (2006) 617–634

14. Dentcheva, D., Prekopa, A., Ruszczynski, A.: Concavity and efficient points ofdiscrete distributions in probabilistic programming. Mathematical Programming,Ser A 89 (2000) 55–77

Integer Programming Approach for Probabilistic Constraints 423

15. Prekopa, A.: On probabilistic constrained programmming. In Kuhn, H.W., ed.:Proceedings of the Princeton Symposium on Mathematical Programming, Prince-ton, N.J., Princeton University Press (1970) 113–138

16. Ahmed, S., Shapiro, A.: The sample average approximation method for stochasticprograms with integer recourse. Preprint available at www.optimization-online.org(2002)

17. Atlason, J., Epelman, M.A., Henderson, S.G.: Call center staffing with simulationand cutting plane methods. Annals of Operations Research 127 (2004) 333–358

18. Shapiro, A., Homem-de-Mello, T.: On the rate of convergence of optimal solutionsof monte carlo approximations of stochastic programs. SIAM Journal of Optimiza-tion 11 (2000) 70–86

19. Ruszczynski, A.: Probabilistic programming with discrete distributions and prece-dence constrained knapsack polyhedra. Mathematical Programming, Ser A 93(2002) 195–215

20. Gunluk, O., Pochet, Y.: Mixing mixed-integer inequalities. Mathematical Pro-gramming 90 (2001) 429–457

21. Atamturk, A., Nemhauser, G.L., Savelsbergh, M.W.P.: The mixed vertex packingproblem. Mathematical Programming 89 (2000) 35–53

22. Guan, Y., Ahmed, S., Nemhauser, G.L.: Sequential pairing of mixed integer in-equalities. Discrete Optimization (Forthcoming, 2007)

23. Miller, A.J., Wolsey, L.A.: Tight formulations for some simple mixed integer pro-grams and convex objective integer programs. Mathematical Programming 98(2003) 73–88

24. Van Vyve, M.: The continuous mixing polyhedron. Mathematics of OperationsResearch 30 (2005) 441–452