LEARNING THE HEURISTIC DISTRIBUTION BY AN EVOLUTIONARY HYPER-HEURISTIC

Edmund Burke, Nam Pham, Rong Qu

Outline

Motivation Constructive hyper-heuristics Heuristic representation Our evolutionary hyper-heuristic

approach An application to graph colouring An application to examination

timetabling Conclusions and Future Work

2

Nam Pham

Motivation

Hyper-heuristics are at higher generality Applicable for different problems without

many modifications Hyper-heuristics may learn effective and

ineffective heuristics It is observed by human that a heuristic

may be better at giving decisions at different stages during the search

Example: ‘largest degree’ and ‘saturation degree’ for graph colouring

3

Nam Pham

Constructive Hyper-heuristics Constructive heuristics

Consist of step-by-step decisions Build a complete solution from scratch

Different problems have different set of decisions at each step

Constructive hyper-heuristics Are heuristics Intelligently choose different constructive

heuristics for different situations

4

Nam Pham

A Heuristic Representation

Sequences of low-level heuristics Each element represents a set of heuristics That set of heuristics provides a number of

decisions to construct one step towards a complete solution

Elements in a sequence are applied consecutively until a complete solution is obtained

We use this representation for problems in this talk.

5

Nam Pham

Our Evolutionary Hyper-heuristic An evolutionary algorithm on the high level

search Chromosomes: sequence representation Divide a sequence into a number of intervals

Learn the appearance frequency of simple low-level heuristics in different intervals of sequences Maintain a list of heuristic probability distribution Update the list by using fittest sequences

With a good probability distribution, we are likely to generate fitter sequences

6

Nam Pham

Our Evolutionary Hyper-heuristic An evolutionary algorithm on the high level

search Chromosomes: sequence representation Divide a sequence into a number of intervals

Learn the appearance frequency of simple low-level heuristics in different intervals of sequences Maintain a list of heuristic probability distribution Update the list by using fittest sequences

With a good probability distribution, we are likely to generate fitter sequences

7

Nam Pham

A Sequence of Heuristics

H1

H1

H1

H1

H2

H3

H1

H1

H2

H3

H2

8

Nam Pham

Intervals of a sequence

H1

H1

H1

H1

H2

H3

H1

H1

H2

H3

H2

9

Nam Pham

Our Evolutionary Hyper-heuristic An evolutionary algorithm on the high level

search Chromosomes: sequence representation Divide a sequence into a number of intervals

Learn the appearance frequency of simple low-level heuristics in different intervals of sequences Maintain a list of heuristic probability distribution Update the list by using fittest sequences

With a good probability distribution, we are likely to generate fitter sequences

10

Nam Pham

List of heuristic probability distribution

H1

H1

H1

H1

H2

H3

H1

H1

H2

H3

H2

H1

100%

H2

0%

H3

0%

H1

33.3%

H2

33.3%

H3

33.3%

H1

0%

H2

66.6%

H3

33.3%

11

Nam Pham

Update the list

H1

H1

H1

H1

H2

H3

H1

H1

H2

H3

H2

H1

100%

H2

0%

H3

0%

H1

33.3%

H2

33.3%

H3

33.3%

H1

0%

H2

66.6%

H3

33.3%

H1

H1

H1

H1

H2

H3

H1

H1

H2

H3

H2

H1

H1

H1

H1

H2

H3

H1

H1

H2

H3

H2

Average Probabilities

12

Nam Pham

Our Evolutionary Hyper-heuristic An evolutionary algorithm on the high level

search Chromosomes: sequence representation Divide a sequence into a number of intervals

Learn the appearance frequency of simple low-level heuristics in different intervals of sequences Maintain a list of heuristic probability distribution Update the list by using fittest sequences

With a good probability distribution, we are likely to generate fitter sequences

13

Nam Pham

Our Hyper-heuristic Outline

Initialise heuristic probability distribution such that all low-level heuristics have the same chance to appear in every interval

Do Sort the evaluation function of all chromosomes Update the list of heuristic distribution using p =

10% of the best fit chromosomes Generate chromosomes for the next generation.

Until stopping condition is met

14

Nam Pham

The Creation of New Chromosomes Keep chromosomes that always select a

particular low-level heuristic Keep the best 10% from the previous generation 10% of the population will be generated

randomly 10% of the population will be generated based

on the probabilities in the list of heuristic distribution

The remaining chromosomes are generated by using a crossover operator Fitter chromosomes are more likely to be selected

15

Nam Pham

Crossover Operator

H1

H1

H1

H1

H2

H3

H1

H1

H2

H3

H2

H1

100%

H2

0%

H3

0%

H1

33.3%

H2

33.3%

H3

33.3%

H1

0%

H2

66.6%

H3

33.3%

Chromosome 1

16

Nam Pham

Crossover Operator

H1

H2

H1

H1

H2

H1

H1

H1

H2

H3

H2

H1

66.6%

H2

33.3%

H3

0%

H1

66.6%

H2

33.3%

H3

0%

H1

0%

H2

66.6%

H3

33.3%

Chromosome 2

17

Nam Pham

Crossover Operator

H1

100%H2

0%H3

0%

H1

33.3%H

233.3

%H3

33.3%

H1

0%H2

66.6%H

333.3

%

H1

66.6%H

233.3

%H3

0%

H1

66.6%H

233.3

%H3

0%

H1

0%H2

66.6%H

333.3

%

Average Probabilities

H1

83.3%H

216.6

%H3

0%

H1

50%H2

33.3%H

30%

H1

0%H2

66.6%H

333.3

%

18

Nam Pham

Generality of our Hyper-heuristic Not designed with only a problem in mind No problem dependent information

Learn from the frequency of low-level heuristics in sequences

Applicable to different problems We test on the graph colouring problem and

the examination timetabling problem The only modifications are the pool of low-

level heuristics and the fitness evaluation for sequences

19

Nam Pham

An Application to Graph Colouring Problem description Pool of low-level heuristics Fitness evaluation for sequences Computational results

20

Nam Pham

Problem description

NP-hard combinatorial optimisation problem (Papadimitriou and Steiglitz, 1982)

Assigning colours to vertices such that adjacent vertices receive different colours

Find a colouring that requires as few colours as possible

21

Nam Pham

Low-level heuristics

Sequence representation Each element in a sequence consists of a vertex-

selection(VS) heuristic and a colour-selection(CS) heuristic

We currently use the same colour-selection heuristic and concern only vertex-selection heuristics

Strategy: a vertex that is likely to cause trouble if deferred until later should be coloured first

VS1

CS

VS1

CS

VS1

CS

VS1

CS

VS2

CS

VS3

CS

VS1

CS

VS1

CS

VS2

CS

VS3

CS

VS2

CS

22

Nam Pham

Pool of low-level heuristics

SD: Least Saturation Degree: Vertices are ordered increasingly by the number of available colours during the colouring process

LD: Largest Degree: Vertices are ordered decreasingly by the number of neighbours in the graph

LCD: Largest Coloured Degree: Vertices are ordered decreasingly by the number of coloured neighbours

SD2, SD3, LD2, LD3, LCD2, LCD3

23

Nam Pham

Fitness Evaluation for Sequences We currently use the evaluation of a

colouring as the sequence evaluation the number of colours required to obtain a

non-conflict colouring breaking ties by the distance to a colouring

using one colour less than the current colouring

24

Nam Pham

Computational Results

Each interval consists of 10 decisions to select heuristics

For each instance, we evolved 5000 generations

Compare our hyper-heuristic with results obtained from other approaches for the Toronto benchmark (Qu et al., 2008)

The probability distribution of heuristics are reported

25

Nam Pham

Computational Results

Instances

Vertices

Best single

heuristic

RandomSequence

s

Our Hyper-

heuristic

Max Clique

Best Reported

Car91 682 30 29 28 23 28

Car92 543 29 28 27 24 28

Ear83 190 23 22 22 21 22

Hec92 81 19 17 17 17 17

Kfu93 461 19 19 19 19 19

Lse91 381 17 17 17 17 17

Rye92 482 22 21 21 21 21

Sta83 139 13 13 13 13 13

Tre92 261 23 20 20 20 20

Uta92 622 31 30 29 26 30

Ute92 184 10 10 10 10 10

Yor83 181 21 19 18 18 19

26

Nam Pham

Heuristic Probability Distribution

27

Nam Pham

Effective and Ineffective Heuristics

28

Nam Pham

Effective and Ineffective Heuristics

29

Nam Pham

An Application to Exam Timetabling Problem description Pool of low-level heuristics Fitness evaluation for sequences Computational results

30

Nam Pham

Problem description

A generalisation of the graph colouring problem Assigning exam to timeslots such that

students do not have to sit two exams at the same time the timetable with the best spread of exams for

students is preferred Using graph colouring to model exam timetabling

vertices represent exams colours represent timeslots

Find a non-conflict timetable that has the smallest penalty for the spread of exams for students

31

Nam Pham

Low-level heuristics

Sequence representation Each element in a sequence consists of a exam-

selection(ES) heuristic and a timeslot-selection(TS) heuristic

We currently use the same timeslot-selection heuristic and concern only exam-selection heuristics

Use the same strategy to select exams as for the graph colouring problem

ES1

TS

ES1

TS

ES1

TS

ES1

TS

ES2

TS

ES3

TS

ES1

TS

ES1

TS

ES2

TS

ES3

TS

ES2

TS

32

Nam Pham

Pool of low-level heuristics

SD: Least Saturation Degree: Exams are ordered increasingly by the number of available timeslots

LD: Largest Degree: Exams are ordered decreasingly by the number of neighbours in the graph

LCD: Largest Coloured Degree: Exams are ordered decreasingly by the number of already assigned neighbours

LE: Largest Enrolment: Exams are ordered decreasingly by the enrolment

LWD: Largest Weighted Degree: Exams are ordered decreasingly by the total number of students in conflict

SD2, SD3, LD2, LD3, LCD2, LCD3, LE2, LE3, LWD2, LWD3

33

Nam Pham

Fitness Evaluation for Sequences If a feasible timetable is found

The evaluation is the average penalty of students sitting in exams close together

If a feasible timetable cannot be found Record the first position, i, that causes infeasibility We use the evaluation as follows

where l is the length of the sequence and controls the degree of involvement for infeasible solutions

),(.)( ilvaluationbestKnownEsf

34

Nam Pham

Computational Results

Each interval consists of 10 decisions to select heuristics

For each instance, we evolved 5000 generations

Compare our hyper-heuristic with results obtained from other constructive approaches for the Toronto examination timetabling benchmark (Qu et al., 2008)

The probability distribution of heuristics are reported

35

Nam Pham

Computational Results

Instances Vertices

OurHyper-

heuristic

Best ReportedConstructi

ve

PercentageDifference

car91 I 682 5.14 4.97 +3.42%

car92 I 543 4.21 4.32 -2.54%

ear83 I 190 35.87 36.16 -0.8%

hec92 I 81 11.65 10.8 +7.87%

kfu93 I 461 14.59 14.0 +4.21%

lse91 381 11.1 10.5 +5.71%

rye92 482 9.55 7.3 +30.8%

sta83 I 139 157.95 158.19 -0.15%

tre92 I 261 8.44 8.38 +0.71%

uta92 I 622 3.43 3.36 +2.08%

ute92 184 27.04 25.8 +4.8%

yor83 I 181 39.07 39.8 -1.83%

36

Nam Pham

Heuristic Probability Distribution

37

Nam Pham

Conclusions and Future Work This evolutionary hyper-heuristic is re-usable for

different problems and requires little problem specific information We will experiment on other problems in future work

Produce solutions of acceptable quality. Some of them are competitive.

The probability distribution of heuristics shows how such heuristics work at different stages of the search

We can learn whether a heuristic is effective or ineffective in different situations

38

Nam Pham

Reference

PAPADIMITRIOU, C. H. & STEIGLITZ, K. (1982) Combinatorial Optimization: Algorithms and Complexity, Prentice-Hall.

QU, R., BURKE, E. K., MCCOLLUM, B., MERLOT, L. T. G. & LEE, S. Y. (2008) A Survey of Search Methodologies and Automated Approaches for Examination Timetabling. to appear in Journal of Scheduling.

39

Nam Pham

Questions/Comments

Thank you!

40

Nam Pham