Multi-Objective Multi-Objective Planning and Planning and

Scheduling with Scheduling with Astronomical Astronomical ApplicationsApplications

Mark Giuliano – Space Telescope Mark Giuliano – Space Telescope Science InstituteScience Institute

Talk Outline

• Why Multi-Objective Planning and Why Multi-Objective Planning and Scheduling?Scheduling?- Motivation Motivation

• Implementation ApproachesImplementation Approaches- Genetic algorithmsGenetic algorithms- Example applicationExample application

• ChallengesChallenges- Visualization tools Visualization tools - Evaluating algorithmsEvaluating algorithms

Overall GoalOverall Goal• Schedule astronomical and other space science Schedule astronomical and other space science

observations to optimize science returnobservations to optimize science return

- Reduce the cost of operations as well as enable more Reduce the cost of operations as well as enable more sciencescience

- Multiple mission phases and granularitiesMultiple mission phases and granularities‣ Mission proof-of-concept versus operationsMission proof-of-concept versus operations‣ Long range planning versus short term schedulingLong range planning versus short term scheduling

- Oversubscribed scheduling Oversubscribed scheduling ‣ More science is approved than time is availableMore science is approved than time is available

- Dynamic environment ― change is the norm not the Dynamic environment ― change is the norm not the exceptionexception‣ Changing science goalsChanging science goals‣ Changing spacecraft capabilitiesChanging spacecraft capabilities

- Multiple often conflicting goals with multiple constituentsMultiple often conflicting goals with multiple constituents‣ Science return, engineering, calibration, stability of the plan itself Science return, engineering, calibration, stability of the plan itself

… … ‣ Hard to quantify and make explicit goals - communication problemHard to quantify and make explicit goals - communication problem

GoalsGoals• Effective decision support tools that enable Effective decision support tools that enable

participants to optimize schedules in a participants to optimize schedules in a collaborative manner...collaborative manner...

GoalsGoals• Effective decision support tools that enable Effective decision support tools that enable

participants to optimize schedules in a participants to optimize schedules in a collaborative mannercollaborative manner- multiple objectivesmultiple objectives

GoalsGoals• Effective decision support tools that enable Effective decision support tools that enable

participants to optimize schedules in a participants to optimize schedules in a collaborative mannercollaborative manner- multiple participantsmultiple participants

GoalsGoals• Effective decision support tools that enable Effective decision support tools that enable

participants to optimize schedules in a participants to optimize schedules in a collaborative mannercollaborative manner- enable integration with existing toolsenable integration with existing tools

Multi-Objective SchedulingMulti-Objective Scheduling• Effective scheduling of space based astronomy missions Effective scheduling of space based astronomy missions

requires the ability to make trade-offs among competing requires the ability to make trade-offs among competing mission objectives:mission objectives:

- Time on target, minimizing use of consumables, Time on target, minimizing use of consumables, minimizing the use of critical mechanisms, preferring minimizing the use of critical mechanisms, preferring the higher priority science, ...the higher priority science, ...

• Objectives are often competing in that improving one Objectives are often competing in that improving one objective means making another objective worseobjective means making another objective worse

- In the short term getting more science done may In the short term getting more science done may decrease the mission lifetimedecrease the mission lifetime

• Objectives have different constituents lobbying for themObjectives have different constituents lobbying for them

- e.g. mission science community versus engineeringe.g. mission science community versus engineering

- E.g. Solar system observers versus galaxy observersE.g. Solar system observers versus galaxy observers

Traditional ApproachTraditional Approach•The traditional approach to handling The traditional approach to handling

multiple objectives is to combine the multiple objectives is to combine the weighted average of separate objectivesweighted average of separate objectives

- ∑ ∑ααii f fii(x)(x)•But: combining objectives loses information But: combining objectives loses information

and pre-determines the trade-offs among and pre-determines the trade-offs among them!them!

• In practice this approach requires users to In practice this approach requires users to run the planning system multiple times each run the planning system multiple times each with different weights for the objectiveswith different weights for the objectives- Users then compare solutions using ad-hoc Users then compare solutions using ad-hoc

methods to select a solution for operationsmethods to select a solution for operations

Multi-Objective Solution Multi-Objective Solution ApproachesApproaches• Multi-Objective Scheduling:Multi-Objective Scheduling:

- Explicitly maintain and exploit multiple Explicitly maintain and exploit multiple objectives during scheduling objectives during scheduling

- Algorithms build up approximate Pareto Algorithms build up approximate Pareto optimal frontier from a population of optimal frontier from a population of candidate schedules candidate schedules ‣ i.e. i.e. ““non-dominatednon-dominated”” solutions, such that no other solutions, such that no other

candidate is better, considering all objectives.candidate is better, considering all objectives.‣ Each point below represents a complete solutionEach point below represents a complete solution

Comparing the ApproachesComparing the Approaches

Traditional Approach Multi Objective Approach• User determines criteria weights

for multiple planner/scheduler runs

• The planner/scheduler is run for each set of criteria/weights

• The results of the runs are compared using ad hoc methods to select a solution for execution

• User performs a single planner/scheduler run to produce a Pareto surface of solutions

• The user explores the Pareto surface to select a solution for execution

• The multi objective approach: The multi objective approach: - Automates steps that users would - Automates steps that users would manually perform in the traditional manually perform in the traditional approach;approach;- Provides a more formal basis to - Provides a more formal basis to select a solution.select a solution.

Tools for Selecting Tools for Selecting SolutionsSolutions• The Pareto frontier gives participants a powerful The Pareto frontier gives participants a powerful

view into the optimal trade-off space, but users view into the optimal trade-off space, but users still need to agree on a particular candidate still need to agree on a particular candidate scheduleschedule

• Need to provide tools that will provide Need to provide tools that will provide distributed decision supportdistributed decision support- Mixed-initiative planning Mixed-initiative planning

‣ support the end user in making trade offssupport the end user in making trade offs

‣ Automate when possible but leave final control Automate when possible but leave final control with the userwith the user

- Graphical internet-based tools that support Graphical internet-based tools that support multiple participantsmultiple participants

- Challenges include: human factors, non-Challenges include: human factors, non-simultaneous users, domain-specific simultaneous users, domain-specific scheduling GUIsscheduling GUIs

Implementation Implementation ApproachesApproaches

• Evolutionary algorithms provide a natural fit Evolutionary algorithms provide a natural fit for finding Pareto-surfacesfor finding Pareto-surfaces- Effective on a wide range of problems- Capable of dealing with objectives that are

not mathematically well behaved (e.g. discontinuous, non-differentiable).

- By maintaining a population of solutions they are capable of representing the entire Pareto frontier at any stage

- Lend themselves to parallelization

Evolutionary AlgorithmsEvolutionary Algorithms

• Based on models of animal EvolutionBased on models of animal Evolution

• Core Algorithm;Core Algorithm;

• Generate the initial populationGenerate the initial population

• Evaluate the fitness of each member of the Evaluate the fitness of each member of the populationpopulation

• Repeat until termination Repeat until termination

- Select the best-fit individuals for reproduction Select the best-fit individuals for reproduction

- Breed New individuals through crossover and Breed New individuals through crossover and mutationmutation

- Evaluate the individual fitness of new individualsEvaluate the individual fitness of new individuals

- Replace least-fit population with new individualsReplace least-fit population with new individuals

GDE3GDE3

• GDE3 is based on GDE3 is based on differential evolutiondifferential evolution optimization for optimization for single-objective problems (Price, et. al 2005)single-objective problems (Price, et. al 2005)

- For each member of the population, select three others For each member of the population, select three others and calculate a candidate child vector by combining the three and calculate a candidate child vector by combining the three vectors using binary crossover and a scaling factorvectors using binary crossover and a scaling factor

- Evaluate the candidate child vector and compare with the Evaluate the candidate child vector and compare with the original population member as follows:original population member as follows:‣ both infeasible: choose less violatedboth infeasible: choose less violated

‣ one feasible, other infeasible: choose feasibleone feasible, other infeasible: choose feasible

‣ both feasible: choose dominating if present, else choose bothboth feasible: choose dominating if present, else choose both

- If necessary, reduce population back to size N via non-If necessary, reduce population back to size N via non-dominated sorting and crowding distance (to improve diversity dominated sorting and crowding distance (to improve diversity along Pareto frontier)along Pareto frontier)

Typical System ArchitectureTypical System Architecture

GDE3 - Implements multi-GDE3 - Implements multi-objective evolutionary objective evolutionary

algorithmsalgorithmsCreates and evolves decision Creates and evolves decision

variable vectorsvariable vectors

SPIKE - Implements scheduling SPIKE - Implements scheduling domain. domain.

Creates and evaluates schedules Creates and evaluates schedules seeded by decision variable seeded by decision variable

vectorsvectors

Application: James Webb Application: James Webb TelescopeTelescope• Launch 2013 Launch 2013

20142014

• Infrared sensors Infrared sensors to detect the to detect the earliest star earliest star formationformation

• L2 orbit 1.5 L2 orbit 1.5 million km from million km from EarthEarth

• 6.2 meter mirror6.2 meter mirror

• Tennis court Tennis court sized sun shield sized sun shield to protect to protect science science instrumentsinstruments

Solar radiation Solar radiation pressurepressure

Sun normal to shield: Sun normal to shield: minimal reaction minimal reaction wheel spin upwheel spin up

Challenge: Challenge: Momentum Momentum SchedulingScheduling

Solar radiation Solar radiation pressurepressure

Sun not normal to shield: Sun not normal to shield: reaction wheel spin to reaction wheel spin to maintain pointingmaintain pointing

Momentum Constraint

•Solar radiation pressure on the sunshield is absorbed as angular momentum in reaction wheels.

•Wheels have a limited momentum capacity.- Momentum dumping requires using non-

renewable fuel to fire thrusters. - Potential limiting factor in the mission

lifetime.

•Momentum accumulation for a target varies over time and the spacecraft roll.- Major factor in the quality of JWST schedules.

JWST Momentum JWST Momentum ChallengesChallenges• The model is a The model is a three dimensionalthree dimensional vector space. vector space.

• Momentum accumulation for an observation Momentum accumulation for an observation varies:varies:

- Over time in Over time in non-linearnon-linear manner manner

• Momentum accumulation is Momentum accumulation is additiveadditive in nature. in nature. - Scheduling an observation at a time can either Scheduling an observation at a time can either

add or subtract from the overall momentum add or subtract from the overall momentum accumulationaccumulation..

• Momentum provides both a Momentum provides both a hard constrainthard constraint due to due to a limited capacity, and a a limited capacity, and a preferencepreference to consume to consume as little resource as possible.as little resource as possible.

JWST Scheduling Objectives

•Minimize Schedule Gaps - JWST Contract mandates no more the 2.5% idle time

•Minimize Momentum Accumulation

•Minimize Observations that miss their last chance to schedule

JWST Scheduling JWST Scheduling Engine (SPIKE)Engine (SPIKE)

JWST Scheduling JWST Scheduling Engine (SPIKE)Engine (SPIKE)

Multi-Objective Multi-Objective Scheduler + Scheduler +

JWST Application JWST Application MapMap

Multi-Objective Multi-Objective Scheduler + Scheduler +

JWST Application JWST Application MapMap

JWST ExperimentsJWST Experiments

• Evaluated system using JWST Science Design Evaluated system using JWST Science Design Reference MissionReference Mission

• Schedule observations to a quantum of 7 minutes in Schedule observations to a quantum of 7 minutes in a 22-day momentum bina 22-day momentum bin

•Using GDE3 evolutionary algorithm (Java) with Lisp-based Using GDE3 evolutionary algorithm (Java) with Lisp-based SPIKE JWST SchedulerSPIKE JWST Scheduler

• Implemented parallel domain scheduler driverImplemented parallel domain scheduler driver

• Candidate vectors are executed in parallel up to the Candidate vectors are executed in parallel up to the population size. population size.

-The Blue and Red dots represent different search decompositions

- Hollow dots represent the Pareto surface.

- The use of parallel evaluations significantly sped up the search process

Experimental ResultsExperimental Results

Evolutionary Algorithm Evolutionary Algorithm FeaturesFeatures• Can seed the initial set of candidate vectorsCan seed the initial set of candidate vectors

- Uniformly creates solutions with fewer dropped Uniformly creates solutions with fewer dropped observations (i.e. observations which miss their last observations (i.e. observations which miss their last chance to schedule)chance to schedule)

• The GDE framework allows constraint limits to be specified The GDE framework allows constraint limits to be specified on criteria valueson criteria values

‣ Will first evolve out of constraint violation spaceWill first evolve out of constraint violation space

‣ Will not consider solutions with violations for crossover if Will not consider solutions with violations for crossover if violation free solutions exitviolation free solutions exit

‣ Want variety in the search space but not at the expense of Want variety in the search space but not at the expense of infeasible solutionsinfeasible solutions

- Need to consider the depth versus the breadth of the Need to consider the depth versus the breadth of the search spacesearch space‣ Is it better to have more generations but less elements in each Is it better to have more generations but less elements in each

generation or to have less generations but more elements in generation or to have less generations but more elements in each generation?each generation?

Selecting Solutions Selecting Solutions • So you have generated a Pareto-Surface of So you have generated a Pareto-Surface of

solutions for a problem, now what do you do?solutions for a problem, now what do you do?

• Still have the problem of selecting a solution Still have the problem of selecting a solution for executionfor execution

• Need to provide tools to end users that enable Need to provide tools to end users that enable them to explore the trade-offs in the Pareto them to explore the trade-offs in the Pareto SurfaceSurface- Trade off space can have a Trade off space can have a high high

dimensionalitydimensionality making it hard for users to making it hard for users to see patterns in the datasee patterns in the data

- Multi-objective problems often require Multi-objective problems often require multiple usersmultiple users to be involved to be involved ‣ Each user contributes one or more objectives Each user contributes one or more objectives

Visualizing the trade-off Visualizing the trade-off spacespace

• Traditional X-Y plots show trade offs between 2 Traditional X-Y plots show trade offs between 2 objectivesobjectives

• Hard to see relationships between the different Hard to see relationships between the different graphsgraphs

• The number of plots increases rapidly as the The number of plots increases rapidly as the number of objectives increasesnumber of objectives increases

Parallel Coordinate GraphsParallel Coordinate Graphs

Parallel Coordinate GraphsParallel Coordinate Graphs•Each solution is represented by a single line

•Creiteria values are plotted horizontally on a normalized scale

•Pros:- Easy to see relationship between the criteria

values of different solutions- Graphs scale linearly with the number of

criteria

•Cons:- Not intuitive in that they need explantions- Graphs can get crowded

Self Organizing MapSelf Organizing Map

Self Organizing MapsSelf Organizing Maps• Colors represent different criteriaColors represent different criteria

• Circles with wedges represent solutionsCircles with wedges represent solutions

• The map conveys information with:The map conveys information with:- The geometry of the color coded shapeThe geometry of the color coded shape

‣ correlation between criteriacorrelation between criteria

- The placement of the circles on the mapThe placement of the circles on the map‣ Correlation with criteria valuesCorrelation with criteria values

- The size of the wedges within a circle The size of the wedges within a circle represent criteria values for solutionsrepresent criteria values for solutions

• The determination of a good self organizing The determination of a good self organizing map is itself a multi-objective optimization map is itself a multi-objective optimization problemproblem

Crowding and Coordinate Crowding and Coordinate PlotsPlots

What Does this HST Plot tell us?What Does this HST Plot tell us?

Interesting SolutionsInteresting Solutions

• Can use existing algorithms to display only the Can use existing algorithms to display only the interesting subset of a Pareto-surface.interesting subset of a Pareto-surface.

• The genetic algorithm used in these The genetic algorithm used in these experiments has a crowding distance measureexperiments has a crowding distance measure- Used to reduce the number of candidates to Used to reduce the number of candidates to

the population size at each generationthe population size at each generation

• We reused the measure to display only the We reused the measure to display only the interesting subset of a Pareto-surface.interesting subset of a Pareto-surface.

A reduced Parallel Coordinate A reduced Parallel Coordinate chartchart

Shows the top 25% most interesting Shows the top 25% most interesting solutionssolutions

What Does this Plot tell us?What Does this Plot tell us?

Exploring the surfaceExploring the surface•No single view of the data is always best

• Interface needs to provide multiple views that allow users to dynamically explore the Pareto surface- Allow users to adjust bounds on what is

displayed- Sort data and or to filter out data- Link together different graphs by selecting

solutions

So you generated a Pareto Surface, Now So you generated a Pareto Surface, Now what? what? Developers ChallengesDevelopers Challenges

• Need tools which allow developers to compare Need tools which allow developers to compare different multi-objective algorithmsdifferent multi-objective algorithms- There are many variants for multiple objective There are many variants for multiple objective

algorithms:algorithms:‣ High level search decomposition (choice of variables, High level search decomposition (choice of variables,

values)values)‣ Parameters controlling searchParameters controlling search

- Number of generations, size of each generationNumber of generations, size of each generation

- Developers want to select the best algorithm Developers want to select the best algorithm variant for their particular application domain.variant for their particular application domain.

Evaluating AlgorithmsEvaluating Algorithms• Features of planning algorithms:Features of planning algorithms:

- Runtime space and time performanceRuntime space and time performance- Ease of system integrationEase of system integration- Transparency (easy to understand results)Transparency (easy to understand results)- Maintainability of the codeMaintainability of the code- Quality of the solutions producedQuality of the solutions produced

• These features apply to both single and multi-These features apply to both single and multi-objective algorithmsobjective algorithms- Evaluating solution quality is different for Evaluating solution quality is different for

multi-objective and single objective multi-objective and single objective algorithmsalgorithms

Evaluating Solution QualityEvaluating Solution Quality• Single objective algorithms produces a single Single objective algorithms produces a single

solution for a problem instancesolution for a problem instance- Maximizes an objective function combining Maximizes an objective function combining

multiple criteriamultiple criteria- Different single-objective algorithms can be Different single-objective algorithms can be

directly compared using the objective directly compared using the objective function function

• Multi-objective algorithm produces a Pareto-Multi-objective algorithm produces a Pareto-surface of solutions where surface of solutions where - No solution is dominated by another solution No solution is dominated by another solution

for all criteriafor all criteria- Comparing algorithms for a problem requires Comparing algorithms for a problem requires

comparing Pareto-surfacescomparing Pareto-surfaces

Evaluating Surfaces • (Zitzler 2003) There is no Unary function, F, on surfaces such that if

F(surface1) > F(surface2) then surface1 is better than surface2

• You can construct binary evaluation functions that detect domination between surfaces

-Let C be the Pareto surface obtained by combining surfaces S1 and S2

-The following function detects domination:

‣If Intersect(S1,C) == S1 and intersect(S2,C)== NULL then S1 dominates S2

‣ Algorithms 1 dominates Algorithm 2

Binary Evaluation Metric 1Binary Evaluation Metric 1

• You can construct binary evaluation functions that give You can construct binary evaluation functions that give metric comparisons between surfaces:metric comparisons between surfaces:

- Define F(S1,S2) = Length(Intersect(S1,C)) / Length(S1) Define F(S1,S2) = Length(Intersect(S1,C)) / Length(S1)

• C is the combined Pareto C is the combined Pareto surface of S1 and S2. surface of S1 and S2.

• In the example In the example

•F(A1,A2) = 4/5F(A1,A2) = 4/5

• F(A2,A1) = 1/5F(A2,A1) = 1/5

Binary Evaluation Metric 2Binary Evaluation Metric 2

• Define E(PDefine E(P11,P,P22) as the factor by which one ) as the factor by which one Pareto surface is worse than another with Pareto surface is worse than another with respect to all objectives. respect to all objectives.

- E(PE(P11,P,P22) is the minimum factor e such that for ) is the minimum factor e such that for any solution in Pany solution in P22 there exists a solution in P there exists a solution in P11 that is not worse by a factor of e in all that is not worse by a factor of e in all objectives. objectives.

- If E(PIf E(P11,P,P22) is smaller than E(P) is smaller than E(P22,P,P11) then the ) then the indicator implies that Pindicator implies that P11 is preferable to P is preferable to P2.2.

Formalism versus GraphicsFormalism versus Graphics

• How well do the binary evaluation metrics How well do the binary evaluation metrics perform compared to intuitions gained from perform compared to intuitions gained from graphic displays of Pareto-surfaces?graphic displays of Pareto-surfaces?- Can we use metrics to evaluate algorithms?Can we use metrics to evaluate algorithms?

• We compare results for two sets of experimentsWe compare results for two sets of experiments- For the purpose of this talk we will just For the purpose of this talk we will just

compare the compare the blueblue algorithm with the algorithm with the redred algorithmalgorithm

Metrics vs Visualization (1)Metrics vs Visualization (1)

Schedule Run

E-metric F-metric

Blue 1.61 0.58

Red 3.45 0.42

• Both of the metrics Both of the metrics indicate a preference for indicate a preference for

the the blue search blue search. . • This agrees with our This agrees with our visual intuition as we visual intuition as we

see more see more blueblue than than redred in the plots.in the plots.

• The metrics do not The metrics do not show features we can show features we can easily see from the easily see from the

charts:charts:• The broad blue search The broad blue search

provides a better min provides a better min gap values at the cost of gap values at the cost of

a high number of a high number of dropped observationsdropped observations

Metrics vs Visualization (2)Metrics vs Visualization (2)

•Both of the metrics Both of the metrics indicate a preference for indicate a preference for the the blue blue search. search. • This agrees with our visual This agrees with our visual intuition as we see more intuition as we see more blueblue than than redred in the plots. in the plots.

•The metrics do not show The metrics do not show features we can easily features we can easily see from the charts:see from the charts:• The delayed search is The delayed search is much better in terms of much better in terms of momentum.momentum.• The two approaches are The two approaches are competitive if we do not competitive if we do not consider momentum.consider momentum.

Schedule Run

E-metric

F-metric

blue 1.58 0.6

red 1.98 0.4

Metrics vs VisualizationMetrics vs Visualization

• Take home thoughts:Take home thoughts:- Metrics can provide a high level comparison Metrics can provide a high level comparison

of Pareto-surfacesof Pareto-surfaces- Metrics miss intuitions that can be gained Metrics miss intuitions that can be gained

through the use of visualization tools.through the use of visualization tools.- Use both techniquesUse both techniques

Evaluating Surfaces - Analogy

• Selecting an evolutionary algorithm out of a set of Selecting an evolutionary algorithm out of a set of algorithms is like selecting a solution for execution algorithms is like selecting a solution for execution out of a Pareto Surfaceout of a Pareto Surface- If there is a dominating algorithm/solution then it If there is a dominating algorithm/solution then it

will be selected will be selected - If there is no domination then it is up to the user to If there is no domination then it is up to the user to

otherwise it is up to user to select a otherwise it is up to user to select a algorithm/solutionalgorithm/solution

• What is needed are tools that allow users to examine What is needed are tools that allow users to examine and manipulate Pareto-Surfacesand manipulate Pareto-Surfaces

ProgrammerProgrammer SchedulerScheduler

Which Which Algorithm Algorithm Should I Should I select?select?

Which Which Schedule Schedule Should I Should I select?select?

OverviewOverview• Multi-Objective algorithms have advantages over Multi-Objective algorithms have advantages over

the traditional approach of combining criteria with the traditional approach of combining criteria with a weighted averagea weighted average- Do not pre-determine trade-offs between criteriaDo not pre-determine trade-offs between criteria- Provide end users with a Pareto surface of Provide end users with a Pareto surface of

solutions to select fromsolutions to select from- Automate manual steps of adjusting criteria Automate manual steps of adjusting criteria

weightsweights

• Existing Multi-Objective Algorithms are effective in Existing Multi-Objective Algorithms are effective in building uniformly sampled approximations of the building uniformly sampled approximations of the Pareto surface Pareto surface

• Challenges in providing dynamic visualization tools Challenges in providing dynamic visualization tools for exploring the surface and selecting solutionsfor exploring the surface and selecting solutions