The W, C, & H Methods 4 Generating Pareto Solutions Prepared by: MA A-Sultan March 20, 2006 Reference: Entropic Vector Optimization and Simulated Entropy:

The W, C, & H Methods The W, C, & H Methods 4 4

Generating Pareto Solutions Generating Pareto Solutions

Prepared by: Prepared by: MA A-SultanMA A-Sultan

March 20, 2006March 20, 2006

Reference: Entropic Vector Optimization and Simulated Entropy: Theory & Applications; Ph.D. Thesis, Liverpool University, 1991.

http://library.liv.ac.uk/search/a?searchtype=t&searcharg=Entropic+Vector+Optimization&searchscope=0&SORT=&SUBMIT=Search

MO OptimizationMO Optimization Methods

The Weighting MethodThe Weighting MethodA 1st standard technique for MOO is to optimize a positively weighted convex sum of the objectives, that is,

Optimize

Not adequate alone! Non-ConvexNon-Convex Pareto solutions can’t be generated; WHY?WHY?

N

iiii XF

1

* 01);(*

N

ii

1

1

For bi-objectivebi-objective example, will create a table like this:

0.8, 0.2

1.0, 0.0

0.0, 1.0

0.2, 0.8

0.35, 0.65

0.225, 0.775

Etc.

Cause Cause Headache?Headache?

What if you What if you have 3, 5, … have 3, 5, … 25 objs.?25 objs.?

Convexity & Concavity

F2(X*)

F1(X*)

Convexity & Concavity Convexity & Concavity

The Weighting Method The Weighting Method

-w1/w2

-w1/w2

-w1/w2

Weights don’t reflect objective importance at all!


The Constraint MethodThe Constraint MethodA 2nd standard technique for MOO is to optimize one objective while setting remaining ones to variable feasible bounds, that is:

Optimize

S.t: iNjbXF

XF

jij

i

,....,2,1;*)(

*)(

jb Values must be chosen so that feasible solutions to the resulting single-objective problem exist.

Cause Cause Headache?Headache?

What if you What if you have 3, 5, … have 3, 5, … 25 objs.?25 objs.?

Virtually, there are weightsweights here too!

The Constraint MethodThe Constraint Method

Effec.

Cost

Specifying a bound on EffecEffec., defines a new

feasible region which does allow CostCost to be minimized

to find a ParertoParerto point.

Generation of Pareto Solutions by Entropy-based Methods

Reference: Entropic Vector Optimization and Simulated Entropy: Theory & Applications; pp.(115 – 118), Ph.D. Thesis, Liverpool University, 1991.


The Entropy-based Weighting Method

The Weighting Method The Entropy-based Weighting Method

N

ii

1

1

N

i

XFP

XFP

ii

i

e

e

1

*)(

*)(

P

01 i

N

iii XF

1

* )(

N

iii

N

iii PXF

11

)ln(1

*)( V =

U =

JjX

V

X

U

jj

;0**

We have proved thatWe have proved that:

where is an optimal solution*jX



Weights follow

Boltzmann’ Distribution!

The Entropy-based Constraint Method

The Constraint Method The Entropy-based Constraint Method

iNjbXF

XF

jij

i

,....,2,1;*)(

*)(

iNjP

XF

PXF

jjjj

iiii

,....,2,1;0)ln(1

*)(

)ln(1

*)(

N

i

XFP

XFP

ii

i

e

e

1

*)(

*)(

PReference: Entropic Vector Optimization and Simulated Entropy: Theory & Applications; pp.(115 – 118), Ph.D. Thesis, Liverpool University, 1991.


Weights follow

Boltzmann’ Distribution!

Bi-Obj OptimizationBi-Obj Optimization Example 1

No more than 2 solutions can be generated since those in-between are CONCAVECONCAVE!

Better Representative Pareto Set


-40

-35

-30

-25

-20

-15

-10

-5

0

0 50 100 150 200 250

F1(X)

F2

(X)


-40

-35

-30

-25

-20

-15

-10

-5

0

0 50 100 150 200 250

F1(X)

F2(

X)



0,

048616*)(

042146*)(

0655.0*)(

..

*)(

)(12*)(

21

2221

213

2212

12

212

11

212

2122

211

XX

XXXXXg

XXXXXg

XXXXg

tS

MaximizeXXXF

MinimizeXXXXXF


Grouped Pareto set with a large GAPGAP between the two!

Better Representative Pareto Set




0

5

10

15

20

25

30

25 30 35 40 45 50 55 60

F1(X)

F2

(X)


0

5

10

15

20

25

30

25 30 35 40 45 50 55 60

F1(X)

F2

(X)

0,

0801610*)(

012*)(

..

*)(

*)(

21

2221

212

211

22

12

2211

XX

XXXXXg

XXXg

tS

MinimizeXXXF

MinimizeXXXF


Large portion of Pareto set is MISSINGMISSING!

The shapeshape of the wholewhole Pareto set is generated




0.3

0.35

0.4

0.45

0.5

0.55

0.2 0.3 0.4 0.5 0.6 0.7

F1(X)

F2

(X)


0.3

0.35

0.4

0.45

0.5

0.55

0.2 0.3 0.4 0.5 0.6 0.7

F1(X)

F2

(X)

0,

040*)(

02.75*)(

0)10096.4(

1078.9180*)(

..

}10

10]

10

1

10096.4

1{[10298.3*)(

)]10()1000()6400([785.0*)(

21

23

22

42

71

6

1

42

8

9314

284

27

52

22

41

2211

XX

XXg

XXg

X

XXg

tS

MinimizeX

XXX

XF

MinimizeXXXXXF


1) A to B Pareto set has no GAP

2) B to C Pareto set has large GAP!

1)B to C Pareto set is better representative!




0

20

40

60

80

100

120

140

160

180

600 700 800 900 1000

F1(X)

F2

(X)

AA

BB

CC


0

20

40

60

80

100

120

140

160

180

600 700 800 900 1000

F1(X)

F2

(X)

AA

BB

CC

0,

0150*)(

020075.025.1*)(

0200*)(

..

*)(

54*)(

21

23

212

211

12

211

XX

XXg

XXXg

XXXg

tS

MaximizeXXF

MaximizeXXXF

Generation of Pareto Solutions by Entropy-based Methods (MOPGP94 ReportMOPGP94 Report)

Portsmouth, UK, 1994

* http://www.amazon.com/gp/product/3540606629/qid=1143034880/sr=1-5/ref=sr_1_5/002-0382939-5305662?s=books&v=glance&n=283155 pp. 164 - 194

The authors present in this paper a method to compute the set of Pareto points for some multi-valued optimization problems. The paper is interestinginteresting and it must be published* since:

1) The Authors apply the entropy method to the study of Pareto points which is a new ideanew idea used also by other authors to the study of scalar optimization problems;

2) The method presented by the authors can be used to compute not only a Pareto point but the set of Pareto points, which is an important aspectimportant aspect;

3) The applications to engineering problems are interestinginteresting.

Generation of Pareto Solutions by the Hybrid MethodHybrid Method

The Hybrid MethodHybrid Method

NMP

XFM

iii

M

iii

;)ln(1

*)(11

Optimize

Subject to:

NMJandiJjP

XF jjjj ;,....,2,1;0)ln(1

*)(

The Entropy-based W, C, & H MethodsW, C, & H Methods

The W-MethodW-Method finds NO ConcaveConcave solutions

The C-MethodsC-Methods finds Convex & ConcaveConvex & Concave solutions

Both Need to be used!

The H-MethodH-Method is far more powerfulpowerful than the W-Method and C-Method; yet rarely yet rarely addressed!addressed!

All 3 Methods3 Methods have NO NO ProblemProblem with # of objectives

How the How the W, C, & HW, C, & H Methods Methods were Incorporated into were Incorporated into

GAME ACSOMGAME ACSOM

Prepared by: Prepared by: Michael M SultanMichael M Sultan

March 28, 2006March 28, 2006


The Weighting MethodThe Weighting MethodA 1st standard technique for MOO is to optimize a positively weighted convex sum of the objectives, that is:

The Constraint MethodThe Constraint MethodA 2nd standard technique for MOO is to optimize one objective while setting remaining ones to variable feasible bounds, that is:

OptimizeOptimize

N

iiii XF

1

* 01);(*

N

ii

1

1 iNjbXF

XF

jij

i

,....,2,1;*)(

*)(

OptimizeOptimize

The Weigh & Constrain Method

Set1 Set2 Set3 Set4 Set5 Set6 Set7 … … Setn

Metric1 Metric1 Metric1 Metric1 Metric1 Metric1 Metric1 Metric1 Metric1 Metric1







... ... ... ... ... ... ... ... ... ...

... ... ... ... ... ... ... ... ... ...

Metricn Metricn Metricn Metricn Metricn Metricn Metricn Metricn Metricn Metricn

The Hybrid Method

Weigh & Constrain Algorithm

SEED

Run SETS

Th

e W

eig

hti

ng

Me

tho

d

Th

e C

on

stra

int M

eth

od

Objective

Constraint

How The Weigh & Constrain Method Work?

"Weigh & Constrain" Algorithm is designed to map the whole decision space in search for none-dominated solutions of continuous or integer multi-objective optimization problems; deterministic or stochastic runs. Here is how it works:

1) For each diagonal Metric, solve a single objective optimization problem2) For each Set of runs, solve a multi objective optimization problem3) Change Seed and Repeate 2

Note, for objective weights and constraint bounds use Uniform distribution.

In Crystal Ball, to change SEED:

1) Click on "Run Preferences" (10th icon from left)2) Click on "Sampling"3) Check "Use Same Sequence of Random Numbers"4) In "Initial Seed Value," select any value between (-infinity, +infinity)

Documents

The W, C, & H Methods 4 Generating Pareto Solutions Prepared by: MA A-Sultan March 20, 2006 Reference: Entropic Vector Optimization and Simulated Entropy: