A Theory-Based Termination Condition for Convergence in ...summerschool/presentations/Deb... · Kalyanmoy Deb Koenig Endowed Chair ... A Theory-Based Termination ... Most Evolutionary

Kalyanmoy Deb Koenig Endowed Chair Professor

Department of Electrical and Computer Engineering

Michigan State University East Lansing, USA [email protected]

http://www.egr.msu.edu/~kdeb

A Theory-Based Termination Condition for Convergence in Real-Parameter Evolutionary Algorithms

Keynote Talk at EVOLVE Conference, Iasi, Romania, 2015


Overview ! Most Evolutionary applications in search and

optimization ! Theoretical Optimality conditions

! Karush-Kuhn-Tucker (KKT) conditions ! KKT conditions for near-KKT points ! Approximate KKT points ! “KKT Proximity” Measure (KKTPM) for single and

multi-objective optimization ! Results on standard test problems ! Termination criterion

! Practicalities of implementation ! A fast and approximate method ! Conclusions

2

3


Evolutionary Optimization (EO): A Mimicry of Natural Evolution and Genetics

begin Solution Representation t := 0; // generation counter Initialization P(t); Evaluation P(t); while not TERMINATION do

P'(t) := Selection (P(t)); P''(t) := Variation (P'(t)); Evaluation P''(t); P(t+1):= Survivor (P(t),P''(t)); t := t+1;

od end

! " Mean approaches optimum ! " Variance reduces ! " No need of gradients

Recombination Mutation

Optimum Initial population

Adopted in Practice !"Nose of Shinkansen (Bullet Train in Japan) N700

Series http://english.jr-central.co.jp/news/n20040616/index.html

!"KONE elevator s control system (http://www.kone.com/countries/en_MP/Documents/Brochures/KONE%20Alta.pdf)

Mitsubishi Regional Jet (MRJ)


4

Termination Condition for EOs •  Maximum # of generations achieved

•  Pragmatic, but no guarantee of near-optimum •  Stagnated condition

•  Past τ gen, f improvement is ≤ ε •  Average and best performance is close

•  A target is achieved •  Pragmatic, but requires knowledge

•  Since none guarantees near-optimum, run EO multiple times •  Can we take help of Optimization Theory?


Together

5

Hybrid Methods are Popular

•  EA followed by local search •  Mutation operator of an EA uses a local search •  How do we even terminate a local search

•  Optimality conditions satisfied? •  Are opt. conditions regular near optimum?

•  Neighbor, monotonic along a line? •  Surprising results follow!

optimum

Keynote Talk at EVOLVE Conference, Iasi, Romania, 2015 6

Unconstrained Single-Objective Problems •  Problem: Minimize f(x) •  Optimality condition: x*={x | Grad f(x) = 0 & H(x)=+ve

definite} Himmelblau Function

50 40 30 20 10 5

2 4 6

x2

x1

500

ï6 ï4 ï2 0 2 4 6ï6ï4ï2

250

0

100 75 60

Norm of Gradient Vector

0 2 4 6

x2

x1

75 100

500 250

ï6 ï4 ï2 0 2 4 6

60

ï6

50 40 30 20 10 5ï4

ï2

Error(x) = ||Grad f(x)||

(4 minima) Error(x) has many zeros, Cannot be used alone Keynote Talk at EVOLVE Conference,

Iasi, Romania, 2015

Monotonicity of Error •  Error reduced to zero locally

If +ve def. check on H(x) is made, four minima are found

Optimal point x=3at y=2

Norm of Gradient Vector

0

400

500

600

700

ï6 ï4 ï2 0 2 4 6N

orm

x1

200

100

300

For unconstrained Problems, norm ||.|| can be used a Proximity Measure locally

Regular, but not so in the entire space Keynote Talk at EVOLVE Conference,

Iasi, Romania, 2015

9


Constrained Optimization Problem

! " Decision variables: x = (x1, x2 ,…, xn) ! " Constraints restrict some solutions to be feasible

! " Equality and inequality constraints ! " Minimum of f(x) need not be constrained minimum ! " Constraints can be non-linear

Min. f(x) s.t. gj(x) # 0 j = 1,2,…,J hk(x)=0 k = 1,2,…,K xi

L $ xi $ xiU i = 1,2,…,n

10

Karush-Kuhn-Tucker (KKT) Conditions

•  A triplet (x, u, v) that sa2sfies above condi2ons is a KKT point, when certain constraint qualifica.on is sa2sfied

Eqm. cond.

Feasibility cond. } Complementary Slackness condition

Non-negativity of Lagrange multiplier

Variable bounds are treated as inequality constraints


11

Geometrical Significance of KKT Conditions

•" 99:"(<=>"6578%257?"

•" !"$(8,6(8"54@(62-("15',257"65=(1"57'A"B$5="%7B(01%4'("15'71>"–"C%D('A"6078%80#("B5$"=%7%=,="

= -

both constr. infeasible

A KKT point need not be a minimum


-

Inequality constraints only

All uj must be +ve or zero

constraint is not satisfied

objective value reduces

12

Karush-Kuhn-Tucker (KKT) Necessity Theorem

•  Let f, g, and h are differen.able func.ons and x* be a feasible solu.on. Let I={j|gj(x*)=0}, a set of ac.ve constraints. Furthermore, grad gj(x*) for j in I and grad hk(x*) are linearly independent. If x* is an op.mal solu.on to the NLP, then there exists a (u*, v*) such that triplet (x*, u*, v*) solves the Kuhn-‐Tucker condi.ons (x* is a KKT point)

•  Linear Independence Constraint Qualifica2on (LICQ) condi2on: (strong CQ) –  Implies certain regularity condi2ons –  Most prac2cal problems sa2sfy the condi2on


13

Main Results

•  KKT necessity theorem can be used to identify points that are not optimal

•  If a feasible point satisfies CQ condition and if it is not a KKT point, it cannot be a minimum

•  If a feasible point satisfies CQ condition and if it is a KKT point, it may or may not be minimum –  Need more conditions to confirm optimality

•  If a feasible point does not satisfy CQ condition, it may or may not be a KKT point or minimum


14

Complementary slackness satisfied

Approximate KKT Point

•" x is "-KKT point, if given ">0, there exists ui#0 for all constraints, such that

•" Proven in Dutta et al. (2012) in J. Global Optimization (Springer Journal)


Dutta, J., Deb, K., Tulshyan, R. and Arora, R. (2013). Approximate KKT points and a proximity measure for termination. Journal of Global Optimization, 56(4), 1463–1499.

Eqm. Cond. not satisfied but within " Complementary slackness satisfied

-KKT point, if given ">0, there exists ui#0 for all constraints, such that

Proven in Dutta et al. (2012) in J. Global Optimization (Springer Journal)

, J., Deb, K., , J., Deb, K., Tulshyan, R. and Arora, R. (2013). Approximate KKT points and a proximity measure for termination. proximity measure for termination. Journal of Global Optimization, 56(4), 1463–1499.

Eqm. Cond. not satisfied but within

15

KKT Error Metric •  A feasible iterate xk found by any opt. algorithm

•  Define KKT Error as “Extent of violation of Equilibrium Condition” for a feasible solution:

•  Then define:

•  At KKT point, KKT Error is zero •  How about at a near-KKT point?

– Surprising results


I(xk): active constraint set

16

A Quadratic Problem with Linear Constraints


Along x=y

Along x+y=3 Acceptable

NOT Acceptable!

17

•  Discontinuity: ui=0 slightly away from boundary, but ui≠0 at boundary -> Sudden change

•  x is a Modified ε-KKT point, if given ε>0, there exists ui≥0 for all constraints, such that

•  Similar theorem proven for smooth and non-smooth problems leading to approximate KKT point


Modified ε-‐KKT Point

Both Eqm. and Complementary slackness cond. are within bounds

Dutta, J., Deb, K., Tulshyan, R. and Arora, R. (2013). Approximate KKT points and a proximity measure for termination. Journal of Global Optimization, 56(4), 1463–1499.

18

Proposed KKT Proximity Metric (KKTPM)

•  For a feasible iterate xk, solve

•  Compute KKTPM = εk

*

•  By-product: Get ui* for every constraint •  Corresponding KKTPM for non-smooth problems defined

with sub-differentials of f and g.


# of constraints

19

A Test Problem


KKT Error Metric

KKT Proximity Metric

KKT Error suddenly zero

KKTPM smoothly goes to zero

A C

A C

Along AC

Along AC

20

G-Series Constrained Test Problems:


g01 g07

g09 g04 KKTPM can be used as termination condition

RGA Results Remarkable, as RGAs do not use any gradient information

21

Comparison with KNITRO


•  KNITRO OptErr requires to find ui

•  Similar reduction in performance metrics

22

Computed Optima and Lagrange Multipliers


Settled once and for all

G-series problems

Refer to J of Global Optimization paper

23


Multi-Objective Optimization: Handling multiple conflicting objectives

! " Bueno, Bonito, Barato?

A doomed car

A Dominated car

3B

Good, nice, cheap

•" Multiple solutions are optimal

•" How to apply KKTPM?

24


Which Solutions are Optimal? Mathematical Definition: A solution x* " X is Pareto-optimal if there is no x"X such that f(x)-f(x*) " -RM

+\{0} Domination-based: ! " x(1) dominates x(2), if

! " x(1) is no worse than x(2) in all objectives

! " x(1) is strictly better than x(2) in at least one objective

! " 1 gets dominated by 3

Min. f1

Min. f2 Objective space

Objective space

25


Pareto-Optimal Solutions ! " P =Non-

dominated(P) ! " Solutions which are

not dominated by any member of the set P

! "O(N log N) algorithms exist

! " Pareto-Optimal set = Non-dominated(S)

! " A number of solutions are optimal

Efficient front


Evolutionary Multi-Objective Optimization (EMO): Principle

Step 1 :

Find a set of Pareto-optimal solutions

Step 2 : Choose one from the set

26

27


Evolution of EMO ! " Early penalty-based approaches ! " VEGA (1984) ! " Goldberg's (1989) suggestion

! MOGA, NSGA, NPGA (1993-95) used Goldberg's suggestion ! Elitist EMO (SPEA, NSGA-II, PAES, MOMGA etc.) (1998 --

Present)

EMOO Web site (as of Jan 2010) 1,534 journal, 2,266 conference 226 PhD theses

NSGA-II NSGA VEGA MOGA

Conference, Iasi, Romania, 2015 NPGA Conference, Iasi, Romania, 2015

SPEA Conference, Iasi, Romania, 2015 MOSES

Weighted Lp norm


Elitist Non-dominated Sorting Genetic Algorithm (NSGA-II) ! NSGA-II

! Modular ! No additional parameter ! Fast

! Commercialization: ! MO-Sherpa (RCT) ! ModeFrontier (Estico) ! iSIGHT (Engeneous) ! VisualDoc (Vanderplatts)

Fast-Breaking Paper in Engineering by ISI Web of Science (Feb 04), Thomson Citation Laureate Award 2006, Current Classic and Most Highly Cited Paper (14,500 GS citations)

(Deb et al., IEEE TEC 2002)

28


NSGA-II Simulation on an Unconstrained Problem

•" Parallel search •" Multiple solutions in a single run

29


NSGA-II on a Constrained Problem

30


Many-Objective Optimization

! Multi-objective: {2,3} objectives ! Many-objective: >3 objectives ! EMO difficulties for many-obj. problems:

1." Large fraction of population gets dominated 2." Maintaining diversity difficult 3." Recombination operator inefficient 4." Representation of PO front requires

exponentially more points 5." Performance measures difficult to compute 6." Visualization is difficult

31

One of the Main Thrusts in EMO Today


NSGA-III (IEEE TEC 2014) An EMO for Many-Obj. Optimization

•" Guided search through supplied reference pts

•" Similar to NSGA-II selection

•" Niching through selecting points close to reference lines •" Normalization •" Association •" Niching No additional parameter needed

32

Some Results of NSGA-III


15-obj. Problem

3-obj. Problem

Constrained Problem

A few Solutions

Constrained Problem

A few

Water Problem

(IEEE TEC August 2014)

33

NSGA-III on ZDT1 (2 Objectives)


NSGA-III on DTLZ2 (3 Objectives)


35

•  315 independently manageable paddocks, each having around 100 choices in any of 10 years •  (100315)10

solutions •  ~1082 atoms in the

universe

Land Use Management After 10 Years

! "sheep substantively lower; ! "dairy drastically reduced; ! "more and less frequent forest harvesting; ! "substantial increase in beef cattle.

Variation of 14 Objectives

KKT Proximity in Evolutionary Multi-objective Optimization

•  As points move towards efficient front, KKTPM must reduce

•  KKTPM must have similar values for points equidistant from front


KKT Optimality Conditions for MO Problems

•  Find λ*,u* for minimum KKT Error:

xk is supplied

Equilibrium condition

Compl. slackness Constraint satisf.

Non-neg. of mult.


KKT Error Metric for Multi-objective Problems

•  An example: •  Solve: using Matlab’s fmincon()


KKT Error on P1

•  KKT Error increases towards efficient front •  KKT Error cannot be a metric



KKT Proximity Metric for MO •  Scalarize MO problem using ASF formulation:

•  Choose z and w •  ASF(KG)=ASF(GH) •  Minimize ASF finds O •  Idea borrowed from MCDM literature

KKTPM (cont.)

•  z: utopian point •  w:

•  If F is on efficient front, it is minimum ASF

•  Otherwise, KKT error will not be zero


•  Smoother ASF:

KKT Proximity Metric (cont.)

o  Recall for 1-‐obj:

o  KKTPM=εk*


KKT Proximity Metric (cont.)

o  Solve and find op2mal εk*

o  Define KKTPM:

1. Relax compl. slackness cond. 2. Add a penalty

Treat ASF as single-obj. problem

Use Matlab’s fmincon() to solve it •  1 linear and 1 quadratic constraints


KKT Proximity Metric on P1

•  Smooth reduction to zero

o  Contour parallel to PO front


ZDT1 Test Problem with NSGA-II

KKTPM Surface NSGA-II Populations N=100

•  KKTPM parallel to PO front •  Population converges as a front


(Zitzler, Deb and Thiele, 2000)

ZDT1 Problem (cont.) •  KKTPM reduces with closeness to PO front •  Correlation to distance (R=0.993)



KKTPM Surface NSGA-II Populations N=100

•  KKTPM parallel to PO front



•  KKTPM is computed for every ND point at every generation of NSGA-II


NSGA-II Populations N=100 •" ZDT4 has multiple local

efficient fronts •" Hence more difficult to

solve •" KKTPM plots

demonstrate this aspect •" Takes 200 generations

to converge Avg g()

Avg. g()

0.4

0.6

0.8

1

0 50 100 150 200 250 300 0

50

100

150

200

250

KK

TPM

Mea

sure

Ave

rage

g()

Generation Number

0

Smallest1st Quartile

Median3rd Quartile

Largest

0.2



DTLZ1 Problems with NSGA-III

3-obj

5-obj

10-obj

KKTPM Variations (NSGA-III)

Avg. g()

0.2

0.4

0.6

0.8

1

0 100 200 300 400 500 600 700 800 900 1000 0

50

100

150

200

250

KK

TPM

Mea

sure

Ave

rage

g()

Generation Number


Median3rd Quartile

Largest

0

(Deb et al., 2002)

DTLZ2 Problems with NSGA-III KKTPM Variations (NSGA-III)

3-obj

5-obj

10-obj

Avg. g()

0.2

0.4

0.6

0.8

1

0 50 100 150 200 0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

KK

TPM

Mea

sure

Ave

rage

g()

Generation Number


Median3rd Quartile

Largest

0


(Deb et al., 2002)

Constrained MO Problems

TNK BNH

60 80 100

1st Quartile

120 140

Median

KK

TPM

Mea

sure

Generation Number

3rd Quartile

0

0.02

Largest

0.04

0.06

0.08

0.1

0.12

0.14

0 20 40

Smallest


(Deb, 2001, Wiley Book)

Problem SRN with NSGA-II

NSGA-II convergence is poor Gen=250

Local search method is being pursued



Problem OSY with NSGA-II

•  25% points did not converge until 250 gen. •  Local search to speed up EMO runs

Local search method is being pursued



Initial Results on Local Search Based Methods •" With and without local search


•" KKTPM identifies non-PO solutions

•" LS helps find true PO points

•

•

KKTPM allows differential treatment of ND solutions

ZDT1

Engineering Design Problems with NSGA-II

Welded Beam Design Car Side Impact Design


(Deb and Jain, 2014, NSGA-III)

KKTPM as Termination Criterion

•  Set a threshold •  Terminate

when it is met •  Consistent

results!

Pros and Cons of KKTPM •" Advantages:

•" Guarantees convergence to theoretical optimum •" Scientific termination condition •" Address troubled areas with local search •" Applicable to classical methods as well

•" Disadvantages: •" Computationally demanding (use after every 10

generations or so) •" Discuss propose a Direct method next

•" Applicable to differentiable problems (use numerical gradients)

Advantages: • Guarantees convergence to theoretical optimum • Scientific termination condition • Address troubled areas with local search • Applicable to classical methods as well

• Disadvantages: • Computationally demanding (use after every 10

generations or so) • Discuss propose a Direct method next


A Computationally Faster Method

"" EH75$("1(6578"6571#$0%7#""" T%78"=%7%=,="5B"3$1#"6571#$0%7#")OR."

#" Q5',257"5B"0"'%7(0$"1A1#(="5B"(<,02571"

"" F01("M?"E8(7260'"#5"O5&#""" F01("U?")R%V6,'#."

#" O!8@"078"OK"(01A"#5"65=&,#("#" O!8@W"OK"W"OR"

#" O!8@W"O5&#"W"OR"

"" X12=0#(8"O(1#"N")O!8@YOK"YOR.Z["

"

" F01("U?")R%V6,'#."# O!8@ 078"OK"(01A"#5"65=&,#("# O!8@W"OK W"OR

# O!8@W"O5&# W"OR

" X12=0#(8"O(1#"N")O!8@YOK YOR.Z[


A Test Case •  Single-var. problem: •  x=0.5 is the optimum •  x=1.0 is not an optimum

•  ε* =0.3441 •  εest=0.3654


x=1.0

ε* =0.0 εest=0.0

61

Comparison with Direct Methods

•  Median KKTPM •  Optimal is in

between Adjusted and Direct

•  Estimated is closer

•  1.2 sec versus 62.2 sec

ZDT1



More Comparisons on ZDT Problems

ZDT2 ZDT4

Estimated is closer to Optimal



More Comparisons on DTLZ Problems

DTLZ1_3 DTLZ1_10

Estimated is closer to Optimal Keynote Talk at EVOLVE Conference,

Iasi, Romania, 2015 64

(Deb et al., 2002)

More Comparisons on DTLZ Problems

DTLZ2_5 DTLZ5_3

Estimated is closer to Optimal Keynote Talk at EVOLVE Conference,

Iasi, Romania, 2015 65

(Deb et al., 2002)

Constrained Bi-Objective Problems

SRN BNH



Practical Problems

CAR WELD


(Deb and Jain, 2014, NSGA-III)

Single-Objective Constrained Optimization (Direct vs. opt.) •  True optimum solution added as the final entry

g01 g07 RGA Method

True Opt. True

Opt.

RGA does not converge

RGA converges well

Stuck


More Single-Objective Constrained Optimization

g10 g18 g24

g02

g04 g08

Poor convergence Good convergence Good convergence

Good convergence Good convergence

Good convergence Easy problem

Stuck

Stuck

Stuck

RGA Method

Comput. Time

•" Approx. methods are much faster

•" Not much loss in accuracy •" Good

Trade-off

S&#>" X12=0#(8" R%$(6#"" K$5@(6#(8" !8@,1#(8"U\[>[" M>]" M>^" M>^" M>]"M[_>]" M>]" M>^" M>^" M>^"U`aa>b" ]>\" ]>\" ]>M" ]>^"U\>U" \>U" \>U" \>U" \>U"][>]" \>^" \>^" \>^" \>^"[[>b" \>U" \>U" \>U" \>U"]U>]" M>U" M>M" M>M" M>M"^[>_" M>M" M>\" M>\" M>\"^]>[" M>U" M>M" M>U" M>U"MU]>b" M>]" M>^" M>^" M>]"[\>]" \>U" \>U" \>U" \>U"UMb>U" M>U" M>U" M>U" M>["a`\>_" b>_" b>M" b>[" b>^"b`_M>`" [\>]" [\>b" [\>a" [M>\"M[^>U" \>_" \>`" \>_" \>_"M\ba>]" [>^" [>^" [>^" [>b"M``>^" M>[" M>[" M>b" M>["U\[>]" M>M" M>\" M>\" M>\"Ub^>]" M>M" M>\" M>\" M>\"b>b" \>M" \>\" \>\" \>\"

MU]M>[" a>]" a>U" a>]" a>]"aM>_" \>^" \>b" \>b" \>^"

M\ab>U" U>`" U>]" U>]" U>a"UU]>]" U>M" U>M" U>U" U>U"


Conclusions ! EMO uses stochastic search principles ! No convergence proof in finite time ! Developed performance metric for

convergence based on KKT optimality conditions

! KKT proximity metric implemented with ASF scalarization (MCDM) method

! Demonstrated to work well on many standard test and engineering problems

! Need to couple with a diversity measure ! Such studies (theory, MCDM and EMO)

should bring respect to EO and EMO fields

71

Relevant Papers •  Deb, K. and Abouhawwash. M. (October, 2014). An Optimality

Theory Based Proximity Measure for Set Based Multi-Objective Optimization. COIN Report No. 2014015. (COIN Website, MSU) http://www.egr.msu.edu/~kdeb

•  Deb, K., Abouhawwash, M., and Dutta, J. (2015). A KKT Proximity Measure for Evolutionary Multi-objective and Many-objective Optimization. Proceedings of Eighth Conference on Evolutionary Multi-Criterion Optimization (EMO-2015). Springer.

•  Deb, K. and Abouhawwash, M. (May, 2015). A Computationally Fast and Approximate Method for Karush-Kuhn-Tucker Proximity Measure. COIN Report No. 2015015.
