On the Normal Boundary Intersection Methodfor Generation of Ecient Front

Pradyumn Kumar Shukla

Institute of Numerical MathematicsDepartment of Mathematics

Dresden University of Technology Dresden PIN 01069, Germanypradyumn.shukla@mailbox.tu-dresden.de

Abstract. This paper is concerned with the problem of nding a repre-sentative sample of Pareto-optimal points inmulti-objective optimization.The Normal Boundary Intersection algorithm is a scalarization scheme forgenerating a set of evenly spaced Ecient solutions. A drawback of thisalgorithm is that Pareto-optimality of solutions is not guaranteed. Thecontributions of this paper are two-fold. First, it presents alternate for-mulation of this algorithms, such that (weak) Pareto-optimality of solu-tions is guaranteed.This improvementmakes these algorithm theoreticallyequivalent to other classical algorithms (like weighted-sum or -constraintmethods), without losing its ability to generate a set of evenly spaced Ef-cient solutions. Second, an algorithm is presented so as to know before-hand about certain sub-problems whose solutions are not Pareto-optimaland thus not wasting computational eort to solve them. The relationshipof the new algorithm with weighted-sum and goal programming method isalso presented.

Keywords: Multi-objective optimization, Ecient front generation,Computationally ecient algorithm.

1 Introduction

There are usually multiple conicting objectives in engineering problems (see forexample [5,12,13,3,11]). Since the objectives are in conict with each other thereis a set of Pareto-optimal solutions. The main goal of multi-objective optimiza-tion is to seek Pareto-optimal solutions. Over the years there have been variousapproaches toward fulllment of this goal. Usually it is not economical to gen-erate the entire Pareto-surface, due to the high computational cost for functionevaluations in engineering problems one aims for nding a representative sampleof Pareto-optimal points. It have been observed that convergence and diversityare two conicting criterions which are desired in any algorithm which tries togenerate the entire ecient front. There are many algorithms for generating thePareto-surface for general non-convex multi-objective problems. See for example[4,10,3] and the references therein.

Most classical generating multi-objective optimization methods use an it-erative scalarization scheme of standard procedures such as weighted-sum or

Y. Shi et al. (Eds.): ICCS 2007, Part I, LNCS 4487, pp. 310317, 2007.c Springer-Verlag Berlin Heidelberg 2007

On the Normal Boundary Intersection Method 311

-constraint method as discussed in [4,10], the drawbacks of most of these ap-proaches is that although there are results for convergence, diversity is hard tomaintain. Thus we see that systematic variation of parameters in these scalar-ization techniques do not guarantee diversity in the solution sets as discussedin ([1]). The Normal Boundary Intersection (NBI) algorithm [2] gave a break-through by using scalarizing schemes that give a good diversity in the objec-tive space. It start from equidistant points on the utopia plane (plane passingthrough individual function minimizers) and then go along a certain direction.The progress along the direction is measured by an auxiliary real variable. Thereare also some other algorithms also which try to generate evenly spaced Ecientsolutions. Weck [7] developed adaptive weighted sum method for multi-objectiveoptimization. Messac [9] developed the Normal Constraint method for gettingeven representation of the ecient frontier.

The developments in this paper are aimed at developing an improved algorithmwhich like direction-based algorithms produces evenly spaced points on the E-cient front and on the other hand like weighted-sum or -constraint method doesnot produce dominated points. This paper also proposes a way to jump over someof non-convex regions in which no Pareto-optimal solution lies. This amounts tosolving a smaller number of sub-problems compared to existing methods.

The rest of the paper is structured as follows: in Section 2, the NBI algorithmis briey described and an improved algorithm is also presented. A method tojump over dominated regions in non-convex cases is also presented. Section 3discusses the relationship of the new algorithm with the weighted-sum and goalprogramming method. Finally conclusions are presented in the last section.

2 NBI Algorithm: Original and a New Formulation

The NBI method was developed by Das et. al. [2] for nding a uniformly spreadPareto-optimal solutions for a general nonlinear multi-objective optimizationproblem. The weighted-sum scalarization approach has a fundamental drawbackof not being able to nd a uniform spread of Pareto-optimal solutions, even if auniform spread of weight vectors are used. The NBI approach uses a scalarizationscheme with a property that a uniform spread in parameters will give rise to anear uniform spread in points on the ecient frontier. Also, the method is inde-pendent of the relative scales of dierent objective functions. The scalarizationscheme is briey described below.

Let us consider the following multi-objective problem (MP):

minxS F (x),where S = {x | h(x) = 0; g(x) 0, a x b}. (1)

Let F = (f1 , f2 , . . . , f

m) be the ideal point of the multi-objective optimization

problem with m objective functions and n variables. Henceforth we shift theorigin (in objective space) to F so that all the objective functions are non-negative. Let the individual minimum of the functions be attained at xi foreach i = 1, 2, . . . ,m. The convex hull of the individual minima is then obtained.

312 P.K. Shukla

The simplex obtained by the convex hull of the individual minimum can beexpressed as , where = (F (x1), F (x

2), . . . F (x

m)) is a m m matrix and

= {(b1, b2, . . . , bm) |m

i=1 bi = 1}. The original study suggested a systematicmethod of setting vectors in order to nd a uniformly distributed set of ecientpoints. The NBI scalarization scheme takes a point on the simplex and thensearches for the maximum distance along the normal pointing toward the origin.The NBI subproblem (NBI) for a given vector is as follows:

max(x,t) t,subject to + tn = F (x),

x S,(2)

where n is the normal direction at the point pointing towards the origin. Thesolution of the above problem gives the maximum t and also the correspondingPareto-optimal solution, x. The method works even when the normal directionis not an exact one, but a quasi-normal direction. The following quasi-normaldirection vector is suggested in Das et al. [2]: n = e, where e = (1, 1, . . . , 1)Tis a m 1 vector. The above quasi-normal direction has the property that NBIis independent of the relative scales of the objective functions.

The main problem with NBI method is that the solutions of the sub-problemsneed not be Pareto-optimal (not even locally). This method aims at gettingboundary points rather than Pareto-optimal points. Pareto-optimal points are asubset of boundary points. These obtained point may or may not be a Pareto-optimal point. Figure 1 shows the obtained solutions (i.e. a, b, c, d, e, f , g, h)corresponding to equidistant points (i.e. A, B, C, D, E, F , G, H) on the convexhull (i.e. line AH).

It can be seen that points d, e and f are not Pareto-optimal but are stillfound using the NBI method. It is because solutions of NBI sub-problem is onlyconstrained to lie on a line, however for any point x, to be Pareto-optimal, theexistence of any point in the objective space which satises F (x) F (x) needto be checked (since such a point dominates F (x)).

Using the concept of the goal attaintment approach of Gembicki [6] we for-mulate a modied algorithm (called as mNBI) in which we solve the followingsub-problem (mNBI) for a given vector is as follows:

max(x,t) t,subject to + tn F (x),

x S,(3)

This change in formulation increases the feasible space so as to check for pointswhich dominate the points on the line. Figure 2 shows the schematics. It can beseen that the points d, e and f which are not Pareto-optimal will not be foundusing the mNBI method. Points x, y and z dominate the corresponding pointsd, e and f (which results in larger value of t) and they are thus not optimal tomNBI . For the optimality of mNBI subproblems we have the following result.

On the Normal Boundary Intersection Method 313

f1

e

B

ED

C

A

fF

gG

hH

a

bc

d

2f

Fig. 1. Schematic showing obtained so-lutions using NBI method

f1

e

B

ED

C

A

F

GH

d

2f

f

g

x

z

y

h

a

bc

Fig. 2. Schematic showing obtained so-lutions using a modied NBI method

Lemma 1. Let (t, x) be the optimal solution of (3) subproblem then x isweakly ecient.

Proof: The proof follows from [6] by using and tn as the goal vector anddirection respectively. unionsqAnother advantage of the mNBI algorithm is that since for non-convex problemsall boundary points need not be Pareto-optimal, for some non-convex regions wecan ignore (jump over the non-ecient regions in the objective space) solvingcertain sub-problems. Figure 3 shows the sketch of our proposed method to this.Suppose we start mNBI from the individual minima A of f1 and move on theline AB (convex hull of individual minima). When we reach point C the optimalsolution of mNBI gives the (weakly) ecient point x. Note that the optimalsolution of the NBI subproblem would have been y. From the constraints inmNBI subproblem we can calculate whether or not the constraints are active atoptima. For example in this case the constraints + tn F (x) are not-activesince the optima x does not lies on the line. Now since the obtained solutionx will always weakly dominate the region x + Rm+ (Lemma 1). Since we knowthe direction n (constant for all sub-problems) we an easily nd the point Don the line AB (convex hull of individual minima, in general a simplex), whichcorresponds to obtained solution x.

Lemma 2. Let (t, x) be the optimal solution of (3). Let w := + tn. Let[S] Rm1 denote the projection (in the direction n) of any surface S Rmonto the simplex obtained by convex hull of individual minima. Then the solutionof (3) for all [(F (x) + Rm+ )

(w Rm+ )] is not Pareto-optimal.

Proof: The proof follows by noting that x weakly dominates the region (F (x)+R

m+ ) and that there exist no feasible point in (w Rm+ ) that dominates w. unionsq

314 P.K. Shukla

f1

2f

A

B

C

D

w

y

x

Fig. 3. Schematic showing how to avoid solving certain sub-problems whose solutionsare not Pareto-optimal

For our example, all subproblems originating from region CD do not give Pareto-optimal solutions and thus we can ignore them. It is to be noted that this infor-mation is obtained before solving the sub-problems. This amounts to solving asmaller number of sub-problems compared to existing methods. This computa-tional gain can be of signicant interest in problems where the cost of functionevaluation is high.

3 Relationship Between mNBI Subproblem andWeighted Sum Method

Given a Pareto-optimal solution x, let h(x) denote the augmented vector ofall constraints that are active at x. We can think of the problem to be con-strained only by h(x). Let w Rm, mi=1 wi = 1, denote positive weights forthe objectives. We consider the weighted linear combination problem as follows

max(x) wF (x),subject to h(x) = 0.

(4)

The problem (4)) will be denoted by LCw. Part of rst-order necessary KKTconditions for optimality of (x, ) for LCw is

xF (x)w + xh(x) = 0. (5)In a similar way, the mNBI subproblem can be written as

min(x,t) t,subject to F (x) tn 0,

h(x) = 0,(6)

On the Normal Boundary Intersection Method 315

Part of rst-order KKT condition for optimality of (x, t, (1), (2)) is

xF (x)(1) + xh(x)(2) = 0,1 n(1) = 0, (7)

where (1) Rm, (1) 0 represents the multipliers corresponds to the in-equality constraints F (x) tn 0, and (2) denote the multipliers of theequality constraints h(x) = 0.

Claim. Suppose (x, t, (1), (2)) is the solution of mNBI andm

i=1 (1)i = 0.

Then LCw with the weight vector w = 1mi=1

(1)i

(1) has the solution

[

x, =1

mi=1

(1)i

(2)]

Proof: Dividing both sides of rst part of (7) by the positive scalarm

i=1 (1)i

and observing that h(x) = 0, the equivalence follows. unionsqHence we can obtain the weighting of objectives that correspond to solution ofmNBI subproblem. Note that if one uses the original NBI method to get theequivalence, since the F (x)tn = 0 are equality constraints the componentsof multiplier (1) Rm can be positive or negative. Thus one could obtainnegative weights [2]. In such a case the solution of NBI subproblem is nonPareto-optimal. This is because irrespective of whether the solution lies in aconvex or a non-convex part of the boundary, it should satisfy the rst-ordernecessary KKT conditions if it is Pareto-optimal. However, this does not occursin mNBI subproblems (the weights are always non-negative).

4 Relationship Between mNBI Subproblem and GoalProgramming

A solution to an mNBI subproblem is also a solution to a goal programmingproblem.

Claim. Suppose (x, t, (1), (2)) is the solution of mNBI andm

i=1 (1)i = 0.

If (1)k is any strict positive component (note that all the components are non-negative), then x solves the following goal programming problem:

min(x) fk(x),subject to fi(x) i, i = k,

h(x) = 0,(8)

with for all i = k the goals given by

i =

{fi(x) if

(1)i = 0;

any nite number fi(x) if (1)i = 0.

316 P.K. Shukla

Proof: Since (x, t, (1), (2)) is the solution of mNBI it satises the rst-order necessary KKT condition (7). Dividing both sides of (7) by (1)k > 0 weobtain

xfk(x)w +i=n

i=1,i=kxfi(x)

(1)i

(1)k

+ xh(x)(2)

(1)k

= 0. (9)

Now with (1)i

(1)k

0 as the multipliers of the m 1 inequality constraints in(8), the goals i satisfy complementarity by denition since

i = fi(x) whenever (1)i = 0,

(1)i

(1)k

(fi(x) i) = 0, i = k. (10)

Using the above together with feasibility of x for (8), we obtain that the point(x,

(1)i

(1)k

, (2)

(1)k

)

satises the rst order necessary optimality conditions of (8).

unionsqNote that as in the case of weighted-sum method if one uses the original NBImethod to get the equivalence, due to the presence of equality constraints F (x) tn = 0, the components of multiplier (1) Rm can be positive or neg-ative. Thus one could obtain negative weights. In such a case the equivalenceof NBI and goal programming problem requires additional assumption that thecomponents of (1) are of the same sign while no such assumptions are neededin the mNBI method.

5 Conclusions

In this paper, we presented an ecient formulation of the NBI algorithm forgetting an even spread of ecient points. This method unlike NBI method doesnot produce dominated points and is theoretically equivalent to weighted-sum or-constraint method This paper also proposes a way to reduce the number of sub-problems to be solved for non-convex problems. We compare the mNBI methodwith other popular methods like weighted-sum method and goal programmingmethods using Lagrange multipliers. It turned out that mNBI method doesnot require any unusual assumption compared to relationship of NBI methodwith weighted-sum method and goal programming method. Lastly we would liketo mention that since some other class of methods like the Normal-ConstraintMethod [9] or the Adaptive Weighted Sum Method [8] use similar line or inclinedsearch based constraint in their sub-problems, the solutions of the sub-problemsof these method are also in general not Pareto-optimal and hence the mNBImethod presented in this paper is superior to them.

On the Normal Boundary Intersection Method 317

Acknowledgements

The author acknowledges the partial nancial support by the Gottlieb-Daimler-and Karl Benz-Foundation.

References

1. I. Das and J.E. Dennis. A closer look at drawbacks of minimizing weighted sum ofobjecties for Pareto set generation in multicriteria optimization problems. Struc-tural Optimization, 14(1):6369, 1997.

2. I. Das and J.E. Dennis. Normal-boundary intersection: A new method for gener-ating the Pareto surface in nonlinear multicriteria optimization problems. SIAMJournal of Optimization, 8(3):631657, 1998.

3. K. Deb. Multi-objective optimization using evolutionary algorithms. Chichester,UK: Wiley, 2001.

4. M. Ehrgott. Multicriteria Optimization. Berlin: Springer, 2000.5. H. Eschenauer, J. Koski, and A. Osyczka. Multicriteria Design Optimization.

Berlin: Springer-Verlag, 1990.6. F. W. Gembicki. Performance and Sensitivity Optimization: A Vector Index Ap-

proach. PhD thesis, Case Western Reserve University, Cleveland, OH, 1974.7. I. Y. Kim and O.de Weck. Multiobjective optimization via the adaptive weighted

sum method. In 10th AIAA/ISSMO Multidisciplinary Analysis and OptimizationConference, 2004.

8. I. Y. Kim and O.de Weck. Adaptive weighted sum method for multiobjectiveoptimization: a new method for Pareto front generation. Structural and Multidis-ciplinary Optimization, 31(2):105116, 2005.

9. A. Messac and C. A. Mattson. Normal constraint method with guarantee of evenrepresentation of complete pareto frontier. AIAA Journal, 42(10):21012111, 2004.

10. K. Miettinen. Nonlinear Multiobjective Optimization. Kluwer, Boston, 1999.11. Andrzej Osyczka. Evolutionary Algorithms for Single and Multicriteria Design

Optimization. Physica Verlag, Germany, 2002. ISBN 3-7908-1418-0.12. Eric Sandgren. Multicriteria design optimization by goal programming. In Ho-

jjat Adeli, editor, Advances in Design Optimization, chapter 23, pages 225265.Chapman & Hall, London, 1994.

13. R. B. Stanikov and J. B. Matusov. Multicriteria Optimization and Engineering.New York: Chapman and Hall, 1995.

IntroductionNBI Algorithm: Original and a New FormulationRelationship Between mNBI Subproblem and Weighted Sum MethodRelationship Between mNBI Subproblem and Goal ProgrammingConclusions