Introduction to the Calculus of variations

7/27/2019 Introduction to the Calculus of variations

http://slidepdf.com/reader/full/introduction-to-the-calculus-of-variations 1/31

CALCULUS OF VARIATIONS IN DISCRETE SPACE

FOR CONSTRAINED NONLINEAR DYNAMIC

OPTIMIZATION

Yixin Chen and Benjamin W. Wah

Department of Electrical and Computer Engineering

and the Coordinated Science LaboratoryUniversity of Illinois at Urbana-Champaign

Urbana, IL 61801, USA

ICTAI 2002, USA

November 4, 2002



Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization

Outline

• Introduction

– Discrete-space dynamic optimization problems

– Existing approaches

• Calculus of variations in discrete time and discrete state space

– Necessary and sufficient conditions for constrained local minima

– Variational search algorithm with node dominance

• Implementations in ASPEN

– Some sample results

• Conclusions

Yixin Chen and Benjamin W. Wah 1



INTRODUCTION



Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Introduction

Dynamic Optimization Problems

Optimization problems with time varying dynamic variables

Control theory and

classical theory of calculus of variations

StateTime

DiscreteContinuous

Continuous

Continuous−state

DiscreteDiscrete−state

Continuous−state

Discrete−time

Continuous−time

Continuous−time

Discrete−time

Discrete−state





Discrete-Time Discrete-State Constrained Dynamic Problems

General Constraints I(Y) = 0

Constraints

G(0) G(1) G(N)

Functional

Objective

F(0) F(1) F(N)

Lagrange

y(0) y(1) y(N) y(N+1)

E(N+1)E(N)E(1)Constraints

E(0)

Local

minimize J [{y( j)}] =N

j=0

F ( j, y( j + 1), y( j)) (1)

s.t. G( j, y( j + 1), y( j)) = 0, j = 0, 1, · · · , N (2)

E ( j, y(y)) = 0, (3)I (Y ) = 0 (4)

where y( j) is defined in discrete space Y ,F

,G

andI

arenot

necessarily continuous or differentiableYixin Chen and Benjamin W. Wah 4




Unconstrained Problems or Problems with Lagrange Constraints

• Path Dominance: Principle of Optimality

– Principle of Optimality in dynamic programming applied on feasible state c

J 2 ≤ J 1 =⇒ P 2 → P 1

s

P 2

P 1

J 2J 1 c

Stage t If c lies on the optimalpath between s and d and

• Polynomial worst-case complexity: O

N |Y|2





Problems with General Constraints

• Path dominance not applicable

– A dominating path may become infeasible due to general constraints

– Exponential worst-case complexity: O

|Y|N

, assuming NP hard

• Node dominance applicable

node

relationdominance

node dominance relations

c2

P 1c1

Termination of c1 by

– Worst-case complexity without path dominance: O

N |Y||s|N

∗ Assuming |s| nodes are not pruned in each stage

∗ Substantially less than O(|Y|N ) when |s| |Y|





Benefits of Using Node Dominance

Dominated nodes at each stage

| s | N + 2

b u n d l e s

s

Utilizing node dominance

Not utilizing node dominance Dominating nodes at each stage

Stage 0 Stage 1 Stage N + 1

Y Y Y

ss

Complexity of each iteration is (N + 2)|Y|

| Y | N + 2

b u n d l e s





Worst-Case Complexities in Path and Node Dominance

Constraint Without General Constraints With General ConstraintsType (Conventional DP) (Without Path Dominance)

Dominance With Path With Path and Without Node With NodeUsed Dominance Node Dominance Dominance Dominance

Complexity O

N |Y|2

O

N |Y| + N |s|2

O

|Y|N

O

N |Y||s|N

Node dominance works well when |s| |Y|




CALCULUS OF VARIATIONS IN DISCRETE SPACE



Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Calculus of Variations in Discrete Space

Constrained Local-Minimum Bundle in Discrete Space

• State vector s in discrete state space Y has user-defined discrete neighborhood

• Discrete neighborhood of bundle Y = {y( j)} is the union of discrete

neighborhoods of all stages, each defined on neighborhoods of states in each stage

Bundle Y

• Bundle Y is a constrained local minimum in discrete space (CLM dn) if

– Y is feasible

– No feasible bundle in N b(Y ) has better functional value than J [Y ]





Solving Overall Problem using Discrete-Space Lagrangian Theory

• First-order necessary and sufficient conditions based on the theory of Lagrangemultipliers in discrete space

Saddle Point Conditions(SP dn)

First-order conditions ⇐⇒

CLM dn ⇐⇒

O(|Y |N )

Overall Search Space





Intuitive Meaning Behind Saddle Points

dynamicallyvarying

(h(x) = 0)

(∆xL(x, λ) = 0)

minx[f (x) + λT H (h(x))]↓

λ↑λ↑

Penalties λ are constraints are satisfied

Gradient descents in x space

Equilibrium point where

to reduce objective function

space to increase penalties

on violated constraints

Gradient ascents in λ

and constraint violations

and objective is minimum





Discrete-Time Discrete-State Euler-Lagrange Equation

• Decompose first-order condition into Discrete-Space Euler-Lagrange Equations

(ELE dn) for each stage

• Decompose SP dn in into N + 2 distributed saddle-point conditions DSP dn

• Distributed necessary and sufficient conditions for CLM dn

¨

©CLM dn ≡ ELE dn ≡ DSP dn

– Only necessary when there are general constraints

• Differences from continuous variational calculus theory

– Do not require differentiability or continuity of functions

– Continuous ELE conditions are only necessary but not sufficient, even in the

absence of general constraints





Solving Distributed Subproblems Using Node Dominance

General Constraints

| s | N + 2

b u n d l e s

s

Y

Stage N+1

| Y | N + 2

b u n d l e s

s s

Y Y

Stage 0 Stage 1





Heuristic Search Procedure For Finding DSP dn

stopY

N

Y

N

Stopping condition

met?

Generate new candidate(s)

in the µ subspace µ loop

Search in

µ subspace?

Start in stage 0 in stage 1

Find SP dn Find SP dn Find SP dn

in stage N + 1

Local Descent in x subspace

Local Ascent in λ subspace

Constrained Simulated Annealing(CSA):




DEMONSTRATIONS ON ASPEN



Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Demonstrations on ASPEN

ASPEN Planner

• Automated Scheduling and Planning Environment at Jet Propulsion Laboratory

• ASPEN models have discrete time horizons

– Each time point is a stage

– Adjacent time points can be collapsed into a single stage

– Current implementation: maximum 100 stages• ASPEN repair/optimization actions provide promising descent directions in

state-variable subspace

– Repair actions: resolve conflicts

– Optimization actions: optimize preferences

• ASPEN does not have the UNDO mechanism





Distributed Lagrangian Formulation

• Assign each conflict ci a unique Lagrange Multiplier λi

• Augmented distributed Lagrangian function of stage t:

Γdn(t) = −ws · Score +

ci∈C (t)

λi ∗ H (ci) +

ci∈C (t)

H (ci)2

2

– ws: weight of score

– Score: preference score of schedule

– C (t): set of conflicts whose time duration intersects with stage t

– H (ci): non-negative value assigned to a conflict reflecting its degree of violation

∗ H (ci) = 1 in current implementation





Distributed Heuristic Search for Finding SP dn in Stage t

• Descent of Γdn(t) in state subspace

– Choose probabilistically from repair actions and optimization actions

– Select random feasible action at each choice point– Apply selected action to current schedule in a child process

– Evaluate Γdn(t) of the new schedule

– Accept new schedule according to Metropolis probability controlled by ageometrically decreasing temperature

– Repeat action in parent process if accepted; otherwise, discard result of child

process (overcome lack of UNDO but limited to 3685 forks)

• Ascent of Γdn(t) in Lagrange-multiplier subspace

λi ←− λi + αiH (ci)




SOME SAMPLE RESULTS



Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Some Sample Results

Benchmark CX1-PREF

• Citizen Explorer-I satellite design and operation planning benchmark

–Multiple competing preferences to be optimized

– Problem generator to generate different problem instances

perl probgen.pl < random seed > < number of orbits >

• ASPEN search setting:

a) Find feasible schedule using repair

b) Optimize score using optimize (default 200 iterations)

c) repeat (a) and (b)





Best Feasible Solution on an 8-Orbit Problem

0.7488

0.749

0.74920.7494

0.7496

0.7498

0.75

0.7502

0 10000 20000

B e s t F e a s i b l e S c o r e

Iteration

ASPENASPEN+DCV+CSA

CSA





Search Progress on an 8-Orbit Problem

• Conflicts vs. Iteration:

0

100

200

300400

500

600

0 10000 20000

N u m b e r o f C o

n f l i c t s

Iteration

ASPEN solving CX1-PREF Benchmark with 8 Orbits

ASPEN

0

100

200

300400

500

600

0 10000 20000

N u m b e r o f C o

n f l i c t s

Iteration

ASPEN+DCV+CSA solving CX1-PREF Benchmark with 8 Orbits

ASPEN+DCV+CSA

• Score vs. Iteration:

0.30.350.4

0.450.5

0.550.6

0.650.7

0.750.8

0 10000 20000

O b j e c

t i v e S c o r e

Iteration

ASPEN solving CX1-PREF Benchmark with 8 Orbits

ASPEN

0.30.350.4

0.450.5

0.550.6

0.650.7

0.750.8

0 10000 20000

O b j e c

t i v e S c o r e

Iteration

ASPEN+DCV+CSA solving CX1-PREF Benchmark with 8 Orbits

ASPEN+DCV+CSA





Best Feasible Solution on a 16-Orbit Problem

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0 10000 20000 30000 40000 50000


Iteration

ASPENASPEN+DCV+CSA

CSA







Best Feasible Solution on OPTIMIZE Benchmark

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0 10000 20000 30000 40000 50000


Iteration

ASPENASPEN+DCV+CSA

CSA





Search Progress on OPTIMIZE Benchmark


0

2

4

6

8

10

12

0 10000 20000 30000 40000 50000

N u m b e r o f C o

n f l i c t s

Iteration

ASPEN solving OPTIMIZE Benchmark

ASPEN

0

2

4

6

8

10

12

0 10000 20000 30000 40000 50000

N u m b e r o f C o

n f l i c t s

Iteration

ASPEN+DCV+CSA solving OPTIMIZE Benchmark

ASPEN+DCV+CSA


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 10000 20000 30000 40000 50000

O b j e c t i v e S c o r e

Iteration

ASPEN solving OPTIMIZE Benchmark

ASPEN0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 10000 20000 30000 40000 50000


Iteration

ASPEN+DCV+CSA solving OPTIMIZE Benchmark

ASPEN+DCV+CSA





Best Feasible Solution on PREF Benchmark

0.48

0.5

0.52

0.54

0.56

0.58

0.6

0 10000 20000 30000 40000 50000


Iteration

ASPEN+DCV+CSAASPEN

CSA


C l l f V D S f C d N l D O S S l R l




Search Progress on PREF Benchmark


0

5

10

15

20

25

30

0 10000 20000 30000 40000 50000

N u m b e r o f C o

n f l i c t s

Iteration

ASPEN solving PREF Benchmark

ASPEN

0

5

10

15

20

25

30

0 10000 20000 30000 40000 50000

N u m b e r o f C o

n f l i c t s

Iteration

ASPEN+DCV+CSA solving PREF Benchmark

ASPEN+DCV+CSA


0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0 10000 20000 30000 40000 50000


Iteration

ASPEN solving PREF Benchmark

ASPEN

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0 10000 20000 30000 40000 50000


Iteration

ASPEN+DCV+CSA solving PREF Benchmark

ASPEN+DCV+CSA


C l l f V i ti i Di t S f C t i d N li D i O ti i ti C l i



Calculus of Variations in Discrete Space for Constrained Nonlinear Dynamic Optimization Conclusions

Conclusions

• Extension of calculus of variations in continuous space to discrete space

• Partitioning general necessary conditions into distributed necessary conditions

• Relying on theory of Lagrange multipliers for discrete constrained optimization

• Significant reduction in the base of the exponential complexity

• Significant improvement in search times with equal or better quality in planning

and scheduling problems as compared to those of ASPEN


Documents

Introduction to the Calculus of variations