Research Article Generalised Adaptive Harmony Search: A

Hindawi Publishing CorporationJournal of Applied MathematicsVolume 2013 Article ID 380985 13 pageshttpdxdoiorg1011552013380985

Research ArticleGeneralised Adaptive Harmony Search A Comparative Analysisof Modern Harmony Search

Jaco Fourie1 Richard Green1 and Zong Woo Geem2

1 Department of Computer Science and Software Engineering University of Canterbury Christchurch 8140 New Zealand2Department of Energy and Information Technology Gachon University Seongnam 461-701 Republic of Korea

Correspondence should be addressed to Zong Woo Geem geemgachonackr

Received 30 January 2013 Accepted 22 March 2013

Academic Editor Xin-She Yang

Copyright copy 2013 Jaco Fourie et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Harmony search (HS) was introduced in 2001 as a heuristic population-based optimisation algorithm Since then HS has become apopular alternative to other heuristic algorithms like simulated annealing and particle swarm optimisation However some flawslike the need for parameter tuning were identified and have been a topic of study for much research over the last 10 years Manyvariants of HS were developed to address some of these flaws andmost of them have made substantial improvements In this paperwe compare the performance of three recent HS variants exploratory harmony search self-adaptive harmony search and dynamiclocal-best harmony search We compare the accuracy of these algorithms using a set of well-known optimisation benchmarkfunctions that include both unimodal andmultimodal problems Observations from this comparison led us to design a novel hybridthat combines the best attributes of these modern variants into a single optimiser called generalised adaptive harmony search

1 Introduction

Harmony search (HS) is a relatively new metaheuristic opti-misation algorithm first introduced in 2001 [1] In thiscontext metaheuristic means that unlike general heuris-tic algorithms that use trial-and-error methods to solve aspecific problem it uses higher level techniques to solve ageneral class of problems efficiently HS fits into the categoryof population-based evolutionary algorithms together withgenetic algorithms (GAs) and the particle swarm optimisa-tion (PSO) algorithm

As is often the case with evolutionary metaheuristicsHS was inspired by a natural phenomenon In this case themethods used by professional musicians to collaborativelyimprovise new harmonies were taken as the inspiration forHS An analogy between improvisation and optimisation wasconstructed and an algorithmwas designed tomimic thewaya musician uses short-term memory and past experiences tolead her to the note that results in the most pleasing harmonywhen played together with the other musicians Harmonysearch is easy to implement and can easily be applied andadapted to solve almost any problem that can be modelled asthe minimisation or maximisation of an objective function

The objective function itself can be continuous or discreteand a smooth gradient is not required No initial solutionsor carefully chosen starting points are required In fact theobjective function is considered a black box by harmonysearch Any procedure that takes a solution vector as inputand gives a fitness score as output can be used as an objectivefunction

These properties make harmony search attractive It hasbeen successfully used in a wide range of disciplines includ-ing computer vision vehicle routing music compositionSudoku puzzle and various engineering disciplines [2ndash8]However it was soon realised that many aspects of HScan be improved and in particular that many of the HSparameters that are often difficult to set can be set andadjusted automatically

An early approach to this automatic adjustment of theHS parameters was the improved harmony search (IHS)algorithm developed by Mahdavi et al [9] They noticed thatthe pitch adjustment rate (PAR) and fret width (FW) (Thefret width (FW) was formally known as the bandwidth (BW)and it is still sometimes referred to as such The terminologyhas since been updated to better reflect the musical analogy)parameters are important to the fine tuning of optimised

2 Journal of Applied Mathematics

solution vectors and they suggested a method of adaptingPAR and FW to the relative progress of the optimiser insteadof keeping these parameters constant through all iterationsTheir algorithm alleviates the problem of choosing an exactvalue for the PAR and FW but it still requires that a range ofvalues specified by aminimum andmaximum value be given

Another early approach to improving HSrsquos performanceand solving the parameter setting problem is the global-bestharmony search (GHS) algorithm [10] It was inspired bythe swarm intelligence concept used in the particle swarmoptimisation algorithm (PSO) [11] Instead of using the FWparameter to adjust possible solutions closer to an optimumGHSmoves component values toward the value of the currentbest solution in the population

Both IHS and GHS improved on the performance of HSand alleviated the parameter setting problem to some degreeHowever just in the last few years many more HS variantswere developed that have shown to be superior to theseearly variants and some require even less manual parametersetting Most of these new variants are inspired by otherevolutionary algorithms and they combine the best aspectsof the simplex algorithm simulated annealing differentialevolution sequential quadratic programming and manyother aspects with HS to form new HS variants [12ndash16] Fora thorough summery of the latest research in HS variants seethe review article by Alia and Mandava [17]

In this paper we concentrate on the comparison ofthree of the most recent and promising HS variantsexploratory harmony search (EHS) self-adaptive harmonysearch (SAHS) and dynamic local-best harmony search(DLHS) All three improve on both the performance of theoriginal HS and many of the earlier HS variants We alsochose these variants because they were not developed ashybrid algorithms by combining HS with other optimisationalgorithms Instead the focuswas on investigating the steps ofHS and adding novelmodifications to key areas of the originalalgorithm

In the section that follows we briefly overview theHS algorithm and explain how the optimiser parametersinfluence the performance of the optimisation process Thisprovides the necessary background to introduce the threeHS variants in the sections that follow In Section 3 wecompare the performance of three HS variants using aset of well-known optimisation benchmark functions Theresults from these tests are then interpreted and discussedin Section 4 These observations then lead us to develop anovel hybrid algorithm called generalised adaptive harmonysearch (GAHS) that we introduce in Section 5 We concludein Section 6 with final remarks and our thoughts on futureresearch in the development of harmony search-based opti-misers

2 Harmony Search and OptimisersBased on Harmony Search

Theharmony search algorithm is ametaheuristic population-based optimisation algorithm inspired by the way musiciansin a band discover new harmonies through cooperative

improvisation [1] The analogy between the optimisationof an objective function and the improvisation of pleasingharmonies is explained by the actions of the musicians in theband Each musician corresponds to a decision variable inthe solution vector of the problem and is also a dimensionin the search space Each musician (decision variable) hasa different instrument whose pitch range corresponds to adecision variablersquos value range A solution vector also calledan improvisation at a certain iteration corresponds to themusical harmony at a certain time and the objective functioncorresponds to the audiencersquos aesthetics New improvisationsare based on previously remembered good ones representedby a data structure called the harmony memory (HM)

This analogy is common to all the HS variants that wewill investigate HS variants differ in the way solution vectors(improvisations) are generated and how the HM is updatedat the end of each iteration We start our explanation of howthis is done differently in each variant with an overview of theoriginal HS algorithm

21 An Overview of Harmony Search The core data structureof HS is a matrix of the best solution vectors called theharmony memory (HM) The number of vectors that aresimultaneously processed is known as the harmony memorysize (HMS) It is one of the algorithmrsquos parameters thathas to be set manually Memory is organised as a matrixwith each row representing a solution vector and the finalcolumn representing the vectorrsquos fitness In an119873-dimensionalproblem the HM would be represented as

[[[[[[[[

[

1199091

11199092

1sdot sdot sdot 119909

119873

1| 1199081

1199091

21199092


119873

2| 1199082

1199091

HMS 1199092

HMS sdot sdot sdot 119909119873

HMS | 119908HMS

]]]]]]]]

]

(1)

where each row consists of 119873 decision variables and the fit-ness score 119908 ([1199091 1199092 119909119873 119908]) Before optimisation startsthe HM is initialised with HMS randomly generated solutionvectors Depending on the problem these vectors can also berandomly chosen around a seed point that may represent anarea in the search space where the optimum is most likely tobe found [3]

The step after initialisation is called improvisation Anew solution is improvised by using three rules memoryconsideration pitch adjustment and random selection Eachdecision variable is improvised separately and any one ofthe three rules can be used for any variable The harmonymemory consideration rate (HMCR) is also one of the HSparameters that must be manually chosen It controls howoften the memory (HM) is taken into consideration dur-ing improvisation For standard HS memory considerationmeans that the decisions variablersquos value is chosen directlyfrom one of the solution vectors in the HM A randomnumber is generated for each decision variable If it is smallerthan the HMCR the memory is taken into considerationotherwise a value is randomly chosen from the range ofpossible values for that dimension

Journal of Applied Mathematics 3

If the HM is taken into consideration the improvisedvalue is chosen randomly from one of the values in the HMThe pitch adjustment rate (PAR) is set during initialisationand it controls the amount of pitch adjustment done whenmemory consideration is used Another random number isgenerated If it is smaller than the PAR the improvised valueis pitch adjusted using

1199091015840

new = 119909new + rand () sdot FW (2)

where 1199091015840

new is the new pitch-adjusted value 119909new is theold value chosen using memory consideration rand() is arandom value between minus1 and 1 and FW is the fret widthparameter The terminology has since been updated to betterreflect the musical analogy [5] that controls the maximumvariation in pitch adjustment and is one of the parameters thatmust be manually set

Once a new value has been improvised the memoryis updated by comparing the new improvisation with thevector in the memory with the lowest fitness If the newimprovisation has a higher fitness it replaces the vector withthe lowest fitness This process of improvisation and updatecontinues iteratively until some stopping criterion is fulfilledor the maximum number of iterations are reached

To summarise the main steps of the HS are as follows

(1) Initialise the HM with possible solutions from theentire search space of the function to be optimised

(2) Improvise a new solution vector using memory con-sideration pitch adjustment and random selection

(3) If the new improvisation is better than the worstsolution in the HM replace the worst solution withthe new improvisation

(4) Check the stopping criteria If it has not been metrepeat Steps (2) and (3) or else proceed to Step (5)

(5) Return the best solution (highest fitness) in the HMas the optimal solution vector of the function

This is a brief overview of HS The interested reader isreferred to [1 5] for a more detailed explanation

22 Exploratory Harmony Search The exploratory harmonysearch (EHS) algorithm was the result of improving theperformance of HS by focussing on the evolution of thepopulation variance in theHMduring optimisationDas et alrealised that the exploratory power of HS can be maximisedby dynamically adjusting the FW to be proportional to thestandard deviation of the HM population [18]

This realisation comes from the following lemma that isproven in [18]

Lemma 1 If the HMCR is chosen to be high (ie very nearto 1) and the FW is chosen to be proportional to the standarddeviation of the HM (ie FW prop 120590(HM) = radicVar(HM)) thenthe expected population variance (without updating the HM)can grow exponentially over iterations

This lemma led them to redefine the FW as a dynamicallyadjusting parameter defined by FW = 119896radicVar(HM) where 119896

is a proportionality constantTheFWis therefore recalculatedat each iteration based on the current HM variance insteadof using a constant value that must be chosen manually

To ensure exponential variance growth the PAR HMCRand 119896 must also be chosen such that ((HMS minus 1)HMS) sdot

HMCR sdot [1 + (13)1198962sdot PAR] is greater than 1 The proof of

this statement can also be found in [18]This simple change to HS results in an algorithm that

is less likely to get stuck in local optima requires lessiterations to reach a desired accuracy and frees the userfrom determining the best value for the FW parameterHowever one could argue that the FW parameter was simplyreplaced by the proportionality constant 119896 In their articleDas et al investigate the effect that variations in 119896 have onthe performance of EHS They determined that EHS is notsensitive to the choice of 119896 and used a constant value (119896 =

117) in all their experimentsThis contrastswith the choice ofthe FW inHS that is usually chosen to fit a particular objectivefunction and can have a significant impact on performance ifchosen poorly

23 Self-AdaptiveHarmony Search In 2010Wang andHuangdeveloped a variant of HS that we refer to as the self-adaptiveharmony search (SAHS) algorithm [19]Theirmain focuswasto alleviate the problem of choosing the best HS parametervalues for a specific problem SAHS does not require theFW and PAR parameters that need to be specified whenusing HS However SAHS like IHS requires that a minimumand maximum value for PAR be specified Specifying aminimum and maximum for PAR is typically much easierthan specifying an exact value and it is more reliable sincethe performance is not as sensitive to changes in the range ofPAR values as it is for a specific constant value

SAHS is somewhat similar to IHS in that it also replacesthe PAR with a range of PAR values Like IHS it alsorecalculates the current value of the PAR by linearly adjustingit between the minimum and maximum values based on theiteration count However it differs in that it decreases the PARby starting at the maximum value and linearly decreasing ituntil the minimum is reached at the final iteration instead ofincreasing it like IHS does

Unlike IHS the FW parameter is not replaced by a rangeof FW values Instead it is replaced by a novel method ofpitch adjustment that does not require an FW parameterWhen pitch adjustment of an improvised decision variableis required it is randomly adjusted for that decision variablebetween the minimum and maximum values found in thecurrent HM using the following equations

1199091015840

119894larr997888 1199091015840

119894+ [max (HM119894) minus 119909

1015840

119894] sdot 119903

1199091015840

119894larr997888 1199091015840

119894minus [1199091015840

119894minus min (HM119894)] sdot 119903

(3)

where 1199091015840119894is the new improvisationmax(HM119894) andmin(HM119894)

are the maximum and minimum values in the HM forthe 119894th decision variables and 119903 is a uniformly generatedrandomnumber between 0 and 1 Each time pitch adjustmentis performed the improvisation is updated by randomlyapplying with equal probability one of these two equations


This approach causes progressively smaller changes to bemade to the new improvisation as max(HM119894) and min(HM119894)converge closer togetherwith increasing iterationsThereforepitch adjustment begins as a rough operator making largerchanges to favour exploration and then becomes a fineoperator favouring exploitation as the optimiser convergescloser to the optimum

Another important difference between SAHS and HS isin the initialisation of the HM In HS the HM is initialisedone decision variable at a time by randomly picking avalue from a uniform distribution of values in the allowablerange However this uniform distribution is only an approx-imation created by scaling the output from a pseudoran-dom number generator to the correct range of values Theresulting distribution is often unevenly distributed causingslow convergence or convergence to local optima Insteadof using a pseudorandom number generator SAHS uses alow discrepancy sequence [20] to initialise the HM Lowdiscrepancy sequences are more evenly spread out in thesearch space than pseudorandom ones causing the initialvectors in the HM to better represent the search space of theproblem

24 Dynamic Local-Best Harmony Search dynamic local-best harmony search (DLHS) was developed by Pan et al asan improvement to IHS and GHS that does not require theHMCR and PAR parameters [21] The focus of DLHS can beroughly divided into three categories First the improvisationstep was improved by basing new improvisations on thecurrent best solution vector in the HM instead of a randomlychosen one Second it divides theHM intomultiple indepen-dently optimised sub-HMs in an attempt to maintain greaterdiversity throughout the search process Third DLHS usesa self-learning parameter set list (PSL) to evolve the HMCRand PAR parameters into the optimal values without any userspecificationThe FW parameter is dynamically adjusted in asimilar way to the method that IHS uses

The modification of the improvisation process is inspiredby GHS Like GHS the random solution vector used inmemory consideration is replaced by the best solution vectorin the HM This tends to concentrate future improvisationsaround the current best solution which is likely a local opti-mum and it may be the global optimum To prevent possibleconvergence around local optima the random solution vectorthat was discarded in memory consideration is instead usedduring pitch adjustment So unlike the pitch adjustment of(2) DLHS uses the following equation

1199091015840

new = 119909rand + rand () sdot FW (4)

where 119909rand is randomly chosen from the current values inthe HM for a particular decision variable

Before improvisation starts the HM is randomly dividedinto 119898 equally sized sub-HMs Each sub-HM then usesits own small subset of solution vectors to independentlyconverge onto a local (or possibly global) optimum Dueto the loss of diversity that comes with restricting the sizeof the individual HMs by splitting them up into sub-HMsit is likely that convergence will not be towards the global

optimum but rather toward a local one To prevent this lossof diversity information is allowed to exchange between sub-HMs with a frequency controlled by the regrouping schedule119877 The regrouping schedule determines how often informa-tion exchange is allowed between sub-HMs by randomlyregrouping the individual solution vectors of all sub-HMsinto 119898 new sub-HM configurations every 119877 iteration Thisregrouping simultaneously increases the diversity of all sub-HMs and allows the best solutions from the entire HM to beshared among sub-HMs

Once the global optimum is identified the injection ofdiversity through the regrouping operation does not furtheroptimise but it can result in inaccurate convergence Forthis reason DLHS enters the final phase of the optimisationprocess after 90 of the iterations have been made In thisfinal phase the regrouping operation is halted and a newHM is formed by combining the best three solution vectorsfrom all sub-HMs into a single new HM The new HM isthen exclusively processed until the maximum number ofiterations has been reached

The HMCR and PAR parameter values are dynamicallydetermined by selecting from a self-learning parameter setlist (PSL)This process starts with the initialisation of the PSLby filling it with randomly generated HMCR and PAR valuesHMCR values are generated from a uniform distributionwhere HMCR isin [09 10] and PAR values are generatedfrom a uniform distribution where PAR isin [00 10] At thestart of each iteration one pair of HMCR and PAR valuesis removed from the PSL and used for that iteration If thecurrent iterationrsquos improvisation resulted in the HM beingupdated the current parameter pair is saved in the winningparameter set list (WPSL) This process continues until thePSL becomes empty The PSL is then refilled by randomlyselecting a pair from theWPSL 75of the time and randomlygenerating a new pair 25 of the time To prevent memoryeffects in the WPSL the WPSL is emptied each time the PSLis refilled The result of this is that the best parameter set isgradually learned and it is specific to the objective functionunder consideration The size of the PSL is set to 200 pairsIt was found that the effect that the size of the PSL has onperformance is insignificant [21]

Unlike the HMCR and the PAR the FW parameter isdynamically adjusted in a way that is similar to the wayIHS adjusts the PAR Like IHS DLHS favours a large FWduring the early iterations to encourage exploration of thesearch space while a small FW is favoured during the finaliterations to better exploit good solutions in the HMThe FWis therefore linearly decreasedwith increasing iterations usingthe following equation

FW (119894) =

FWmax minusFWmax minus FWmin

MI2119894 if 119894 lt

MI2

FWmin otherwise(5)

where FWmax and FWmin are the maximum and minimumvalues of the FW 119894 is the iteration number and MI is themaximum number of iterations Like IHS this means thata minimum and maximum value for the FW has to be setbefore optimisation starts but this is again assumed to bemuch easier than deciding on a single value for the FW


221

100minus1

minus1minus2

minus2

minus3 minus3

119884

119883

2000400060008000

100001200014000

1

10

100

1000

10000

Figure 1 Rosenbrockrsquos valley is illustrated here in two dimensionsNotice the parabolic valley that includes the global optimum at[1 1]

3 Performance Analysis Using FiveBenchmark Functions

31 Benchmark Functions We start our performance analysiswith a brief description of the five benchmark functionsthat we will be using to compare performance All five ofthese functions were designed to be challenging to optimisefor different reasons and all have previously been used forbenchmarking harmony search-based algorithms

We start with a classic parametric optimisation functioncalled Rosenbrockrsquos valley [22] This multidimensional prob-lem has a global optimum inside a long narrow parabolic-shaped flat valley The valley itself is easy to find since thereare no other local minima anywhere in the search space butconverging onto the global optimum is difficult Rosenbrockrsquosvalley is defined by the following equation

119891 (x) =

119899minus1

sum

119894=1

100 (119909119894+1

minus 1199092

119894) + (1 minus 119909

119894)2

minus 2048 le 119909119894le 2048

(6)

Rosenbrockrsquos valley has a global optimum at 119909119894= 1 forall119894 where

the function evaluates to 0 An illustration of Rosenbrockrsquosvalley implemented in two dimensions is seen in Figure 1

Rastriginrsquos function uses cosine modulation to create ahighly multimodal function that often causes optimisationalgorithms to get stuck in the local optima without everfinding the global optimum [23] However it does have asingle global optimum at 119909

119894= 0 forall119894 where the function

evaluates to 0 Rastriginrsquos function is defined by

119891 (x) = 10119899 +

119899

sum

119894=1

(1199092

119894minus 10 cos (2120587119909

119894)) minus 512 le 119909

119894le 512

(7)

minus4

minus44

4

minus2minus2

2

2

00 10

10

20

2030

30 4040

5050

6060

70

70

80

80

119884

119883

Figure 2 This illustration of Rastriginrsquos function shows its highlymultimodal nature and it is difficult to visually identify the globaloptima at [0 0]

minus150

minus150

minus100

minus100minus50 minus500

0

50

50

100

100024681012

15

3

45

6

75

9

105

12

119884

119883

550minus100


0

50

50

100

1010

119884

119883

Figure 3 This is a two-dimensional surface plot of Griewangkrsquosfunction The global optima are at [0 0] where the functionevaluates to 0

An illustration of this function in two dimensions is seen inFigure 2 One can clearly see the multiple local minima in thecontours overlaid on the floor of the surface plot

Griewangkrsquos function is similar to Rastriginrsquos functionin that it also uses cosine modulation to create a highlymultimodal function [23] However Griewangkrsquos functionis optimised over a larger search space and has differentproperties as one gets closer to the global optima situated at119909119894= 0 forall119894When one considers the entire search space the function

looks like a simple multidimensional unimodal parabolawhich could easily be minimised using gradient descent Asone moves closer to the optimum and starts to considersmaller areas it becomes clear that the function is not assmooth as first thought and it is in fact highly multimodalThis is clearly illustrated in Figure 3 where a surface plot ofa two-dimensional implementation of Griewangkrsquos function


is shown The multimodal nature of Griewangkrsquos function isalso apparent from its defining equation which is as follows

119891 (x) =

119899

sum

119894=1

1199092

119894

4000minus

119899

prod

119894=1

cos(119909119894

radic119894

) + 1

minus 600 le 119909119894le 600

(8)

Ackleyrsquos Path function is another multimodal test func-tion that is widely used to benchmark optimisation algo-rithms [24] It uses a combination of exponentials and cosinemodulation to create a search space with many local optimaand a single global optimum at 119909


evaluates to 0 Ackleyrsquos Path function is illustrated in Figure 4and is defined by the following equation

119891 (x) = 20 exp(minus02radic1

119899

119899

sum

119894=1

1199092

119894)

minus exp(1

119899

119899

sum

119894=1

cos (2120587119909119894)) + 20 + 119890

minus 32768 le 119909119894le 32768

(9)

The last test function that we consider is the Goldstein-Price function This eighth-order polynomial is defined intwo variables only and has global minima at [00 minus10]wherethe function evaluates to 30 This function has four localminima that lie close together and the search space is usuallylimited to the minus2 le 119909

1 1199092

le 2 cube The Goldstein-Price function is illustrated in Figure 5 and is defined by thefollowing polynomial

119891 (x) = (1 + (1199091+ 1199092+ 1)2

times (19 minus 141199091+ 31199092

1minus 14119909

2+ 611990911199092+ 31199092

2))

times (30 + (21199091minus 31199092)2

times (18 minus 321199091+ 12119909

2

1+ 48119909

2minus 36119909

11199092+ 27119909

2

2))

minus 32768 le 119909119894le 32768

(10)

32 Test Results We compared the performance of the threeHS variants together with the original HS algorithm usinga series of test runs over all five benchmark functions Weimplemented all the benchmark functions except for theGoldstein-Price function in both 30 and 100 dimensions tomeasure performance in both low- and high-dimensionalproblems In functions with 100 dimensions the algorithmswere allowed to optimise for 500000 function evaluationswhile those in 30 dimensions were allowed to run for 100000The Goldstein-Price polynomial is only defined in twodimensions and therefore requires considerably less iterationsto optimise We allowed only 10000 function evaluationswhen the Goldstein-Price polynomial was optimised

In our first experiment all HM variants were imple-mented using the parameters suggested by the original

minus15

minus15

minus10

minus10

minus5 minus500

5

5

10

10

5

10

15

20

25

5

75

10

125

15

175

20

119884

119883

Figure 4 This is a two-dimensional surface plot of Ackleyrsquosfunction The global optima is at [0 0] where the function evaluatesto 0

minus2 minus2minus15

minus15

minus1minus1

minus05minus05

00

0505

11

1515

200000400000

600000

800000

10

100

1000

10000

100000

119884

119883

Figure 5 The Goldstein-Price polynomial is only defined in twovariables and is shown here evaluated in the minus2 le 119909

1 1199092le 2 cube

authors The only exceptions were the parameter sets usedin the original harmony search algorithm and that usedfor DLHS Many different optimal parameter sets have beensuggested forHS by researchers andwe chose to find our ownset based on the best overall results after simulating all of fivebenchmark functions For DLHS the authors suggest 3 sub-HMsof size 3which ismuch smaller than the 50 elementHMssuggested by all the other HM variants [21] We thereforeopted to use 5 sub-HMs of size 10 instead both because thisresulted in better results for DLHS and also for a more faircomparison with the other variants The parameter sets thatwe used in all our experiments are indicated in Table 1

Each of the five benchmark functions were minimisedusing these parameters The results for the 30 dimensionalproblems (together with the Goldstein-Price problem) areshown in Table 2 In each experiment three values are givenThe first is the average score calculated over 50 independentlyinitialised runsThis is followed by the standard deviation and


Table 1 Parameter sets as suggested by the original authors

HS EHS DLHS AHSHMS 5000 5000 5000 5000HMCR 099 099 mdash 099PAR 033 033 mdash mdashPARmin mdash mdash mdash 000PARmax mdash mdash mdash 100FW 001 mdash mdash mdashFWmin mdash mdash 00001 mdashFWmax mdash mdash (UB minus LB)200dagger mdash119896 (EHS) mdash 117 mdash mdash119877 (DLHS) mdash mdash 5000 mdash119898 (DLHS) 500daggerUB and LB refer to the lower and upper bound of the component values

then the success rate defined as the number of successful hitson the global minimum within a 001 tolerance

Notice that in many cases more than one algorithmachieved 100 success rate (shown as 50 successful hits)on the global minimum In these cases we define the bestresults as the algorithm with both a 100 success rate andan average score closest to the global minimum We repeatthis experiment using the same parameter values on 100-dimensional problems The average scores from the secondexperiment are shown in Table 3

Some of these values seem surprising and it is worthinvestigating why this is so One would expect that the 2-dimensional Goldstein-Price function would be the easiestto minimise and that all the average scores should be veryclose to 30 which is the global optimum However as seenin Table 2 this is not the case for HS DLHS and EHS Thesealgorithms are not necessarily so much worse than AHS butthis indicates that some of the 50 runs that contributed tothe average score converged to a local minimum that wasfar different from the global one Since these local minimamay be much larger than the global minimum it can have alarge effect on the average scoreWe see this clearly happeningin the graph of Figure 6 The graph shows that only 2 of the50 runs failed to converge to the exact global minimum butbecause of these 2 the average score is significantly differentthan the global minimum

A quick comparison of the results presented in Tables2 and 3 suggests that the AHS algorithm is the overall bestoptimiser In the 30-dimensional experiment it performedthe best on two of the five benchmark functions and wasthe only optimiser to achieve a 100 success rate on theGoldstein-Price polynomial Its performance in optimisingAckleyrsquos Griewankrsquos and Rosenbrockrsquos functions is verysimilar to that of EHS and there are no significant differencesthat might indicate that one is clearly better than the otherwhen only these three functions are considered Howeverboth completely failed at minimising Rosenbrockrsquos functionIt is intersting that the only successful run from all algorithmswhen optimising Rosenbrockrsquos function was achieved bythe original unmodified HS algorithm This shows that the

0 10 20 30 40 50 600

5

10

15

20

25

30

35

Runs

Opt

imal

val

ue

Optimiser progress

Run optima

Figure 6 This bar graph shows the result of 50 independent runsof EHS on the Goldstein-Price polynomial Notice that 48 of the 50runs converged to the correct globalminimumwhile two convergedto one of the local minima that is 10 times larger than the globalminimum

original HS algorithm can perform equally well and some-times even better than modern variants when an optimisedparameter set is used

Surprising results are also seen in Rastriginrsquos column ofTable 2 Like with Rosenbrock function both EHS and AHSfailed to optimise this function but unlike Rosenbrockrsquos theDLHS optimiser performs well here and has a much betterhit ratio than any of the other variants In the section thatfollows we suggest a possible explanation for this result andinvestigate how one can use this to design an improved HSvariant

The results from the 100-dimensional problem sum-marised in Table 3 are very similar to that of the 30-dimensional problem For Ackleyrsquos Griewankrsquos and Rosen-brockrsquos functions the performance of EHS and AHS isagain the best and is not significantly different Like inthe 30-dimensional problem both EHS and AHS failed inoptimising Rastriginrsquos function which is again optimisedmuch more successfully by DLHS

4 Interpretation of Results

In this section results from Tables 2 and 3 are used to showthe best attributes of the HS variants and where they fail Inthe previous section we saw that EHS and AHS generallyperform the best but both failed to optimise Rastriginrsquos func-tion DLHS performsmuch better on Rastriginrsquos function buttends to get stuck in local optima as seen in the results from30-dimensional Ackleyrsquos and the Goldstein-Price functionsThe question is then why does DLHS perform somuch better


Table 2 Average scores and standard deviations over 50 runs for 30-dimensional problems All optimisers were allowed to run for 100000function evaluations except when the Goldstein-Price polynomial was being minimised In that case 10000 evaluations were allowed Thebest results from each benchmark function are indicated in bold

Ackley Griewank Rosenbrock Rastrigin Goldstein-Pricedagger

HSBest 00066 00322 307287 00142 68464Standard deviation 00006 00334 226459 00027 93353Hits 50 48 1 2 42

DLHSBest 08464 00170 327372 02003 62444Standard deviation 05173 00180 228117 04200 87724Hits 2 50 0 31 44

EHSBest 00008 00006 289183 154119 40809Standard deviation 00024 00022 107493 108267 52910Hits 50 50 0 5 48

AHSBest 00003 00042 265041 14819 3Standard deviation 00020 00111 05624 09055 0Hits 50 50 0 1 50

daggerGoldstein-Price was implemented in two dimensions as it is only defined in two

Table 3 Average scores and standard deviations over 50 runs for 100-dimensional problemsThe optimisers were allowed to run for 500000function evaluations

Ackley Griewank Rosenbrock Rastrigindagger

HSBest 00142 00050 1625829 02280Standard deviation 00004 00080 495829 00130Hits 50 34 0 0

DLHSBest 02281 00004 1424515 13140Standard deviation 00130 00070 395126 31364Hits 50 50 0 32

EHSBest 00009 00001 972555 3882894Standard deviation 00006 00007 74711 205949Hits 50 50 0 0

AHSBest 00003 0 964333 67471Standard deviation 00006 0 02943 23279Hits 50 50 0 0

daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos function

than EHS and AHS on Rastrigin when it is worse in all otherexamples

We start to answer that question by investigating whatnovel improvement these HS variants contribute to the finalresult The main contribution of both EHS and AHS isthe dynamic adjustment of the aggressiveness of the pitch

adjustment operator EHS does this by dynamically adjustingthe value of the FW based on the variance of the valuescurrently in the HM AHS also determines the amount ofpitch adjustment by analysing the values in the HM but itdoes so without needing to directly calculate the varianceIn both cases the effect is that pitch adjustment is more


aggressive at the beginning of the optimisation process andcontinues to make smaller and smaller adjustments as thevariance in the HM decreases due to convergence

The advantage that each of these approaches brings isthat the pitch adjustment operator dynamically shifts from anaggressive mode that favours exploration of the search spaceat the start of optimisation to a less aggressive mode thatfavours exploitation of possible minima that were discoveredduring the aggressive mode However this also means thatwhen the optimiser converges to local minima the drop invariance of values in the HM causes the pitch adjustment tofavour exploitation making it unlikely that the optimiser willescape the local minima and find the global one Prematureconvergence to local optima is therefore a weakness of thesemethods and may explain why even the original HS scoredbetter than EHS and AHS on the optimisation of Rastriginrsquosfunction

AHS attempts to minimise the effect of this weaknessusing the adjustment of the PAR By linearly decreasing thePAR from a very high to a very low value the explorationphase is lengthened due to the aggressive and frequent pitchadjustment caused by a high PAR and a high variance in theHM at the start of optimisation In practice this does have apositive effect on the results and may explain the advantageAHS had over EHS in the Goldstein-Price experiment It washowever still not enough to prevent the poor performancewhen optimising Rastriginrsquos function

DLHS uses a completely different approach first max-imising the HM diversity by dividing it up into sub-HMs andthen by dynamically adjusting the PAR FW and HMCR val-ues to fit the function being optimised and the optimisationprogressThe sub-HM idea has been used successfully beforeto maximise diversity in harmony search (see [4 25]) Thisidea of separating the population of candidate solutions intogroups that converge independently is also used successfullyin other evolutionary algorithms and it was originally usedin genetic algorithms in what became known as island modelparallel genetic algorithms [26 27]

The weakness of this approach however is often slowconvergence due to function evaluations that are wasted insubpopulations (sub-HMs) and end up converging to localoptima or not converging at all Since the subpopulationsconverge independently it is likely that they will convergeto different optima This is actually the desired behavioursince the motivation of this approach was the maximisationof diversity in the candidate solutions However the questionthen becomes when to consolidate the independent resultsfrom each subpopulation so the best candidates can beexploited for more accurate convergence

The answer to this question lies in the classic tradeoffbetween maximising the convergence speed and minimisingthe probability of premature convergence to local optimaIf sub-populations converge independently for too longiterations are wasted on results that will finally be discardedOn the other hand when results are consolidated too quicklydiversity among the sub-populations is lost and this increasesthe risk of premature convergence

In DLHS this tradeoff is left as a parameter namelythe regrouping schedule and it is up to the user to decide

how aggressively the sub-HMs should be consolidated It isdifficult to choose the best value for this parameter and theoptimum value is likely problem specific This means that asingle good choice is unlikely The original authors of DLHSsuggested a value of 50 iterations for the regrouping scheduleRastriginrsquos function was one of the functions they used to testtheir algorithm [21] This may explain the good results thatDLHS produced on Rastriginrsquos function

Another major contributing factor to the DLHS resultsis the dynamic adjustment of the PAR FW and HMCRparameters Like EHS andAHS DLHS starts with a large FWand decreases it as the optimiser progresses However unlikeEHS andAHS it does not take the variance in the currentHMinto account but instead linearly decreases it proportionallyto the iteration count This is similar to the way the IHSalgorithm operates (see [9]) and to the way AHS linearlydecreases the PARWe believe that not taking the variance inthe HM into account is a weakness of DLHS especially sincethe consolidation of the sub-HMs can have a dramatic effecton the HM causing changes in the variance that do not followa linearly decreasing trend

However the PAR and HMCR are adjusted in a novelway that potentially has the largest impact on the quality ofthe final results To our knowledge DLHS is also the onlyHM variant to dynamically adjust the HMCR The HMCRis a parameter that has previously been shown by severalresearchers including the authors of AHS and EHS to havea large effect on the quality of the final results [10 18 19]

In our own analysis (see Section 24) of the values that endin theWPSL after several thousand iterations we noticed thatthe self-adapting process that controls the PAR and HMCRvalues tends to favour very small PARs (lt001) and largeHMCRs (gt099) This seems to verify our choice of optimalvalues for the HMCR and the PAR but it also suggests thatthe maximum PAR used in AHS should optimally be muchsmaller than 10

Finally one should also be very aware of the fact thatbecause the HM is divided into sub-HMs the actual HMimprovisation steps are performed using a much smallerHMS than the 50 candidates that were chosen for theother HM variants Like the HMCR HMS also makes alarge difference to the quality of the results but it is oneof the hardest parameters to choose [10 19] Researchersinitially recommended a small HMS (lt15) but the authorsof AHS showed that AHS performs better with a largerHMS which was also the conclusion made by the authors ofEHS However in our own experiments we found that AHSperforms much better on Rastriginrsquos function even to thepoint of being better than DLHS when the HMS is decreasedto 10 instead of the suggested 50 However this caused adecrease in performance when optimising Griewankrsquos andAckleyrsquos functions This then suggests that the HMS shouldbe chosen to fit the problem or it should be dynamicallyadjusted like the PAR and FW

It is currently not known how the HMS should beadjusted or modelled to a specific problem The practicaleffects that this would have on theHMwould bemuch greaterthan simply changing theHMCRor the PAR since candidateswill need to be added or removed from the HM A further


problem would then be determining which candidate shouldbe removed when the HMS is decreased and how shoulda new candidate be created when the HMS is increasedIf we are to develop an HM variant that performs well onall our benchmark functions without requiring a fine-tunedparameter set unique to each problem this is the type ofquestion that will need to be answered

5 Generalised Adaptive Harmony Search

Our approach to designing an HS variant that performswell on all our benchmark functions is based on the AHSalgorithm We call it the generalised adaptive harmonysearch (GAHS) algorithm We chose to base GAHS on AHSbecause of its simplicity and its good performance on all butRastriginrsquos function As we have pointed out AHS can bemade to perform well on Rastrigin given the right parameterset but no single parameter set seems to perform well on allthe benchmark functions

We therefore need to either find a way to dynamicallyadjust the parameters during optimisation to fit the particularproblem (particularly the HMS) or augment the improvisa-tion process some other way that would allow for the samebenefits Our first attempt at addressing this was to splitthe HM into sub-HMs in the same way that DLHS doesIt was our hope that this would both decrease the effect ofHMS which we already know has a positive effect on theresults for Rastriginrsquos function and increase diversity in theHMenough to compensate for AHSrsquos tendency for prematureconvergence We used the AHS pitch adjustment operatorand linearly decreased the PAR with the intention that thiswould compensate for DLHSrsquos slow convergence

Our initial results showed that using sub-HMs in AHSimproved the performance when minimising Rastriginrsquosfunction but the convergence rate also slowed considerablyresulting in poor performance when minimising GriewankandAckley Several parameter sets were triedwhich indicatedthat a longer regrouping period was needed to compensatefor AHSrsquos tendency for premature convergence Larger sub-HMs resulted in good performance in Griewank and Ackleywhile Rastrigin needed small sub-HMs like those DLHS usesHowever no single parameter set that performed well on allour benchmark functions could be found

A better approach was needed and it was clear that usingsub-HMs will either result in slow convergence or the needto fine-tune a parameter set to a specific problem The focusshould be on the HMS parameter as this parameter has thegreatest effect on the performance of AHS when Rastriginrsquosfunction is minimised

We propose to find a small collection of parameter setsin particular HMS and HMCR values that includes at leastone set that will result in good performance on any ofour benchmark functions An instance of AHS will thenbe started for each parameter set and run for a fraction ofthe total function evaluations allowed Once this has beendone for each parameter set the best results from eachHM is compared and the instance with the worst result

is dropped The process then repeats until only the bestinstance with the most optimal parameter set remains Thisinstance is then left to converge until the remainder of theallowable function evaluations have been reached All AHSinstances are initialised using a lowdiscrepancy sequence (seeSection 23) and regroupings of candidates between separateinstances are never done Each instance therefore convergesindependently

This approach requires the addition of two importantparameters The first called the period length (PL) is thenumber of iterations that a particular instance is allowed torun before it is stopped and evaluated to determine whethermore function evaluations are spent on it or whether it isabandoned The second is the parameter set collection size(PSCS) which is the number of parameter sets in the collec-tion and is therefore also the number of AHS instances thatare compared These two parameters are important becausetogether they determine the percentage of the total functionevaluations wasted on finding the optimal parameter set andhow many are left for convergence to the global optimumSince one function evaluation is done during each iterationof AHS the total number of iterations spent on instances thatdo not contribute to the final result (wasted iterations) can becalculated using the following equation

Wasted iterations = PLPSCSsum

119894=2

(119894 minus 1) (11)

To maximise the number of useful iterations the PL and thePSCS is kept as low as possible However if the PSCS are toosmall an important parameter set may be overlooked If thePL is too small some of the AHS instances may be discardedbefore an accurate estimate of their quality can be measured

Through experimentation we determined that an opti-mal parameter set collection contains three parameter setsAll three parameter sets are equal to that suggested by theoriginal authors of AHS and are identical except for theHMSWe use three different values for theHMS namely 10 30 and50 A good value for the PL was found to be 10 of the totalnumber of iterationsThismeans according to (11) that 30ofthe total iterations is spent on finding the optimal parameterset and 70 for convergence to the global optimum

The performance of GAHS as defined in the previousparagraphs is compared with the other HS variants inTable 4 The benchmark functions were again implementedin 100 dimensions We repeat the results from Table 3 forcomparison

It is seen from this comparison that GAHS performs evenbetter than DLHS on Rastriginrsquos function as well as AHSon the other benchmark functions It still performs poorlyon Rosenbrockrsquos function Even when we used a fine-tunedparameter set specific to Rosenbrockrsquos function we could notimprove the resultsWe suspect that since EHS also convergesto this point that large local optima exist in that area andthat the nature of this function is such that escaping fromthis local optimum is very difficult for harmony search-basedoptimisers

It is also clear that some weaknesses that hinder GAHSstill remain Since GAHS is only a generalised form of AHS


Table 4 In this table the performance ofGAHS is comparedwith the otherHS variants All scores are averaged over 50 runs and all functionsexcept Goldstein-Price are implemented in 100 dimensions The optimisers were allowed to run for 500000 function evaluations except forGoldstein-Price which was allowed only 10000 evaluations The best results are again indicated in bold

Ackley Griewank Rosenbrock Rastrigindagger Goldstein-Price





GAHSBest 00263 00049 964034 00633 3Standard deviation 00125 00021 07833 00337 0Hits 50 49 0 44 50

daggerThe accuracy tolerance for awarding a successful hit on the global minimum was decreased to 01 for 100-dimensional Rastriginrsquos functionGoldstein-Price was implemented in two dimensions as it is only defined in two

one would expect that it would perform at least as wellas AHS in all examples However as we see in Table 4with Ackleyrsquos and Griewankrsquos functions that this is not trueThe reason for this is that AHS enters a slow convergencephase once the basin of attraction around local optima isfound because of the drop in HM variance that causesthe pitch adjustment operator to make smaller and smalleradjustments This means that AHS requires several thousanditerations to find the exact optimum once it has found thebasin of attraction It therefore uses all the available iterationsand keeps converging until the very end If 30of its availableiterations is removed as is done in GAHS the final result willnot be as accurate as it would have been otherwise

Another weakness that affects the convergence speedis in the choice of the period length The period lengthhas to be long enough so that an accurate estimate can bemade of the performance of the parameter set This canonly be an estimate since the true performance may onlybecome apparent once an instance has fully converged andwe already know that AHS keeps converging even in the lastfew iterations This means that our method of measuringthe quality of a parameter set is inherently biased to thosesets that perform well at the beginning of the optimisationprocess This means that those sets with a small HMS anda large HMCR will frequently get chosen causing valuableparameter sets that may later produce the best results to bediscarded during the first round of eliminations

An illustration of this effect is shown in Figure 7 Thegraph traces the convergence of GAHS over 500000 itera-tions as it minimises Ackleyrsquos function The fitness score ofthe best and the worst candidates that the optimiser currentlyhas in the HM is recorded at each iteration and shown astwo separate lines on the graph We see a downward trendindicating convergence followed by discontinuous jumps thatindicate the abandonment of one instance of AHS and thestart of another Note that there are three distinct convergencepaths present in the first 150000 iterations These representthe three AHS instances that correspond to three parametersets The first one that clearly converges the quickest over thefirst 50000 iterations corresponds to the parameter set wherethe HMS = 10 The two that follow are those that correspondto the HMS = 40 and HMS = 50 parameter sets Since weknow from the previous experiments that AHS performsbetter than GAHS and uses an HMS of 50 we know thatGAHS would finally give the best results if the third instancewas kept through both elimination rounds However due toits slow rate of convergence during the first 50000 iterationsit is eliminated first leaving a weaker parameter set to finallyconverge

6 Conclusions

After investigating the results from comparing some of thebest performing HS variants available today we noticed that


0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress

This is the HMS = 50 instance that gets eliminated

HM maximumHM minimum

Iterations

Figure 7This graph traces the convergence of the GAHS algorithmover 500000 iterations as it minimises Ackleyrsquos function Thefitness value which is simply the candidate evaluated using Ackleyrsquosfunction of the best and worst candidates in the HM is indidcatedat the end of each iteration The best candiate is the HM minimumindicated by the blue line the worst candidate is the HMmaximumindicated by the red line

the overall quality of these algorithms was similar We alsonoticed several weaknesses that prevent any one of themfrom being the best choice Authors of these HS variantssuggested parameter sets that are nearly optimal howeversome problems still required the parameters to be fine-tunedto a specific problem if good results were to be expected

We suggested possible ways that some of these weak-nesses can be addressed and we proposed the generalisedadaptive harmony search (GAHS) algorithm as a generalisedversion of AHS GAHS performs well over a larger range ofproblems By using 5 benchmark functions we demonstratedthat GAHS performs as well as AHS on most problems andit can also produce good results on problems where AHS failsto find the global optimum

However some open questions still remain as a topic offuture research Our experiments clearly pointed out thatthere are categories of optimisation problems and that aparameter set that works well for one might fail completelyfor another one For example whatmakesRastriginrsquos functionso difficult to minimise using a parameter set that workswell for all the other benchmark functions What is specialabout Rastriginrsquos function that causes it to require a smallHM to effectively optimise and can this special attribute bedetected before optimisation is attempted A similar questioncan be asked about Rosenbrockrsquos function Why can noneof the HM variants find the global optimum even whenusing a fine-tuned parameter set Is there a weakness thatis inherent in harmony search-based optimisers that makethem poor optimisers for that category of problems to whichRosenbrockrsquos function belongs or can one augment harmony

search in some way to address this weakness Harmonysearch is still a relatively recent addition to the family ofevolutionary algorithms and finding answers to these typeof questions is needed if harmony search is to become fullyestablished as a leading heuristic optimiser

Acknowledgment

This work was supported by the Gachon University researchfund of 2013 (GCU-2013-R084)

References

[1] Z W Geem J H Kim and G V Loganathan ldquoA new heuristicoptimization algorithm harmony searchrdquo Simulation vol 76no 2 pp 60ndash68 2001

[2] O M Alia R Mandava D Ramachandram and M E AzizldquoDynamic fuzzy clustering using harmony search with appli-cation to image segmentationrdquo in Proceedings of the 9th IEEEInternational Symposium on Signal Processing and InformationTechnology (ISSPIT rsquo09) pp 538ndash543 December 2009

[3] J Fourie S Mills and R Green ldquoHarmony filter a robustvisual tracking system using the improved harmony searchalgorithmrdquo Image and Vision Computing vol 28 no 12 pp1702ndash1716 2010

[4] J Fourie RGreen and SMills ldquoCounterpoint harmony searchan accurate algorithm for the blind deconvolution of binaryimagesrdquo in Proceedings of the International Conference onAudioLanguage and Image Processing (ICALIP rsquo10) pp 1117ndash1122Shanghai China November 2010

[5] ZWGeemRecent Advances inHarmony SearchAlgorithm vol270 of Studies in Computational Intelligence Springer BerlinGermany 1st edition 2010

[6] Z W Geem ldquoHarmony search algorithm for solving Sudokurdquoin Knowledge-Based Intelligent Information and EngineeringSystems B Apolloni RHowlett and L Jain Eds LectureNotesin Computer Science pp 371ndash378 Springer Berlin Germany2010

[7] Z W Geem K S Lee and Y Park ldquoApplication of harmonysearch to vehicle routingrdquoAmerican Journal of Applied Sciencesvol 2 no 12 pp 1552ndash1557 2005

[8] ZW Geem and J-Y Choi ldquoMusic composition using harmonysearch algorithmrdquo in Proceedings of the EvoWorkshops pp 593ndash600 Springer Berlin Germany

[9] M Mahdavi M Fesanghary and E Damangir ldquoAn improvedharmony search algorithm for solving optimization problemsrdquoAppliedMathematics and Computation vol 188 no 2 pp 1567ndash1579 2007

[10] M G H Omran and M Mahdavi ldquoGlobal-best harmonysearchrdquo Applied Mathematics and Computation vol 198 no 2pp 643ndash656 2008

[11] J Kennedy and R Eberhart ldquoParticle swarm optimizationrdquo inProceedings of the International Conference on Neural Networkspp 1942ndash1948 December 1995

[12] M Fesanghary M Mahdavi M Minary-Jolandan and YAlizadeh ldquoHybridizing harmony search algorithmwith sequen-tial quadratic programming for engineering optimization prob-lemsrdquoComputerMethods inAppliedMechanics and Engineeringvol 197 no 33ndash40 pp 3080ndash3091 2008


[13] Z W Geem ldquoParticle-swarm harmony search for water net-work designrdquo Engineering Optimization vol 41 no 4 pp 297ndash311 2009

[14] H Jiang Y Liu and L Zheng ldquoDesign and simulation of sim-ulated annealing algorithm with harmony searchrdquo in Advancesin Swarm Intelligence Y Tan Y Shi and K Tan Eds LectureNotes in Computer Science pp 454ndash460 Springer BerlinGermany 2010

[15] W S Jang H I Kang and B H Lee ldquoHybrid simplex-harmonysearch method for optimization problemsrdquo in Proceedings ofIEEE Congress on Evolutionary Computation (CEC rsquo08) pp4157ndash4164 June 2008

[16] L P Li and L Wang ldquoHybrid algorithms based on harmonysearch and differential evolution for global optimizationrdquo inProceedings of the 1st ACMSIGEVO Summit on Genetic andEvolutionary Computation (GEC rsquo09) pp 271ndash278 ACM NewYork NY USA June 2009

[17] OMAlia andRMandava ldquoThevariants of the harmony searchalgorithm an overviewrdquo Artificial Intelligence Review vol 36no 1 pp 49ndash68 2011

[18] S Das A Mukhopadhyay A Roy A Abraham and B K Pan-igrahi ldquoExploratory power of the harmony search algorithmanalysis and improvements for global numerical optimizationrdquoIEEETransactions on SystemsMan andCybernetics Part B vol41 no 1 pp 89ndash106 2011

[19] C M Wang and Y F Huang ldquoSelf-adaptive harmony searchalgorithm for optimizationrdquo Expert Systems with Applicationsvol 37 no 4 pp 2826ndash2837 2010

[20] H Faure ldquoGood permutations for extreme discrepancyrdquo Jour-nal of Number Theory vol 42 no 1 pp 47ndash56 1992

[21] Q K Pan P N Suganthan J J Liang and M F Tasgetiren ldquoAlocal-best harmony search algorithm with dynamic subpopula-tionsrdquo Engineering Optimization vol 42 no 2 pp 101ndash117 2010

[22] H Rosenbrock ldquoAn automatic method for finding the greatestor least value of a functionrdquoThe Computer Journal vol 3 no 3pp 175ndash184 1960

[23] A Torn and A PilinskasGlobal Optimization Lecture Notes inComputer Science 350 Springer Berlin Germany 1989

[24] D H Ackley A Connectionist Machine for Genetic Hillclimbingvol SECS28 of The Kluwer International Series in Engineeringand Computer Science Kluwer Academic Publishers BostonMass USA 1987

[25] J Fourie S Mills and R Green ldquoVisual tracking using the har-mony search algorithmrdquo in Proceedings of the 23rd InternationalConference Image and Vision Computing New Zealand (IVCNZrsquo08) Queenstown New Zealand November 2008

[26] T Niwa and M Tanaka ldquoAnalysis on the island model parallelgenetic algorithms for the genetic driftsrdquo in Selected papers fromthe Second Asia-Pacific Conference on Simulated Evolution andLearning on Simulated Evolution and Learning (SEALrsquo98) vol98 pp 349ndash356 Springer London UK 1999

[27] B Artyushenko ldquoAnalysis of global exploration of island modelgenetic algorithmrdquo in Proceedings of the 10th InternationalConference on Experience of Designing and Application of CADSystems inMicroelectronics (CADSM rsquo09) pp 280ndash281 February2009

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of


Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of


Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of


CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of


Operations ResearchAdvances in

Journal of


Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences


The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Algebra

Discrete Dynamics in Nature and Society



Decision SciencesAdvances in

Discrete MathematicsJournal of


Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of


solution vectors and they suggested a method of adaptingPAR and FW to the relative progress of the optimiser insteadof keeping these parameters constant through all iterationsTheir algorithm alleviates the problem of choosing an exactvalue for the PAR and FW but it still requires that a range ofvalues specified by aminimum andmaximum value be given

Another early approach to improving HSrsquos performanceand solving the parameter setting problem is the global-bestharmony search (GHS) algorithm [10] It was inspired bythe swarm intelligence concept used in the particle swarmoptimisation algorithm (PSO) [11] Instead of using the FWparameter to adjust possible solutions closer to an optimumGHSmoves component values toward the value of the currentbest solution in the population

Both IHS and GHS improved on the performance of HSand alleviated the parameter setting problem to some degreeHowever just in the last few years many more HS variantswere developed that have shown to be superior to theseearly variants and some require even less manual parametersetting Most of these new variants are inspired by otherevolutionary algorithms and they combine the best aspectsof the simplex algorithm simulated annealing differentialevolution sequential quadratic programming and manyother aspects with HS to form new HS variants [12ndash16] Fora thorough summery of the latest research in HS variants seethe review article by Alia and Mandava [17]

In this paper we concentrate on the comparison ofthree of the most recent and promising HS variantsexploratory harmony search (EHS) self-adaptive harmonysearch (SAHS) and dynamic local-best harmony search(DLHS) All three improve on both the performance of theoriginal HS and many of the earlier HS variants We alsochose these variants because they were not developed ashybrid algorithms by combining HS with other optimisationalgorithms Instead the focuswas on investigating the steps ofHS and adding novelmodifications to key areas of the originalalgorithm

In the section that follows we briefly overview theHS algorithm and explain how the optimiser parametersinfluence the performance of the optimisation process Thisprovides the necessary background to introduce the threeHS variants in the sections that follow In Section 3 wecompare the performance of three HS variants using aset of well-known optimisation benchmark functions Theresults from these tests are then interpreted and discussedin Section 4 These observations then lead us to develop anovel hybrid algorithm called generalised adaptive harmonysearch (GAHS) that we introduce in Section 5 We concludein Section 6 with final remarks and our thoughts on futureresearch in the development of harmony search-based opti-misers

2 Harmony Search and OptimisersBased on Harmony Search

Theharmony search algorithm is ametaheuristic population-based optimisation algorithm inspired by the way musiciansin a band discover new harmonies through cooperative

improvisation [1] The analogy between the optimisationof an objective function and the improvisation of pleasingharmonies is explained by the actions of the musicians in theband Each musician corresponds to a decision variable inthe solution vector of the problem and is also a dimensionin the search space Each musician (decision variable) hasa different instrument whose pitch range corresponds to adecision variablersquos value range A solution vector also calledan improvisation at a certain iteration corresponds to themusical harmony at a certain time and the objective functioncorresponds to the audiencersquos aesthetics New improvisationsare based on previously remembered good ones representedby a data structure called the harmony memory (HM)

This analogy is common to all the HS variants that wewill investigate HS variants differ in the way solution vectors(improvisations) are generated and how the HM is updatedat the end of each iteration We start our explanation of howthis is done differently in each variant with an overview of theoriginal HS algorithm

21 An Overview of Harmony Search The core data structureof HS is a matrix of the best solution vectors called theharmony memory (HM) The number of vectors that aresimultaneously processed is known as the harmony memorysize (HMS) It is one of the algorithmrsquos parameters thathas to be set manually Memory is organised as a matrixwith each row representing a solution vector and the finalcolumn representing the vectorrsquos fitness In an119873-dimensionalproblem the HM would be represented as

[[[[[[[[

[

1199091

11199092


119873

1| 1199081

1199091

21199092


119873

2| 1199082

1199091

HMS 1199092

HMS sdot sdot sdot 119909119873

HMS | 119908HMS

]]]]]]]]

]

(1)

where each row consists of 119873 decision variables and the fit-ness score 119908 ([1199091 1199092 119909119873 119908]) Before optimisation startsthe HM is initialised with HMS randomly generated solutionvectors Depending on the problem these vectors can also berandomly chosen around a seed point that may represent anarea in the search space where the optimum is most likely tobe found [3]

The step after initialisation is called improvisation Anew solution is improvised by using three rules memoryconsideration pitch adjustment and random selection Eachdecision variable is improvised separately and any one ofthe three rules can be used for any variable The harmonymemory consideration rate (HMCR) is also one of the HSparameters that must be manually chosen It controls howoften the memory (HM) is taken into consideration dur-ing improvisation For standard HS memory considerationmeans that the decisions variablersquos value is chosen directlyfrom one of the solution vectors in the HM A randomnumber is generated for each decision variable If it is smallerthan the HMCR the memory is taken into considerationotherwise a value is randomly chosen from the range ofpossible values for that dimension



1199091015840


where 1199091015840























1199091015840

119894larr997888 1199091015840

119894+ [max (HM119894) minus 119909

1015840

119894] sdot 119903

1199091015840

119894larr997888 1199091015840

119894minus [1199091015840

119894minus min (HM119894)] sdot 119903

(3)








1199091015840








FW (119894) =


MI2119894 if 119894 lt

MI2

FWmin otherwise(5)



221

100minus1

minus1minus2

minus2

minus3 minus3

119884

119883

2000400060008000

100001200014000

1

10

100

1000

10000





119891 (x) =

119899minus1

sum

119894=1

100 (119909119894+1

minus 1199092

119894) + (1 minus 119909

119894)2

minus 2048 le 119909119894le 2048

(6)






119891 (x) = 10119899 +

119899

sum

119894=1

(1199092

119894minus 10 cos (2120587119909

119894)) minus 512 le 119909

119894le 512

(7)

minus4

minus44

4

minus2minus2

2

2

00 10

10

20

2030

30 4040

5050

6060

70

70

80

80

119884

119883


minus150

minus150

minus100


0

50

50

100

100024681012

15

3

45

6

75

9

105

12

119884

119883

550minus100


0

50

50

100

1010

119884

119883







119891 (x) =

119899

sum

119894=1

1199092

119894

4000minus

119899

prod

119894=1

cos(119909119894

radic119894

) + 1

minus 600 le 119909119894le 600

(8)





119899

119899

sum

119894=1

1199092

119894)

minus exp(1

119899

119899

sum

119894=1

cos (2120587119909119894)) + 20 + 119890

minus 32768 le 119909119894le 32768

(9)


1 1199092


119891 (x) = (1 + (1199091+ 1199092+ 1)2

times (19 minus 141199091+ 31199092

1minus 14119909

2+ 611990911199092+ 31199092

2))

times (30 + (21199091minus 31199092)2

times (18 minus 321199091+ 12119909

2

1+ 48119909

2minus 36119909

11199092+ 27119909

2

2))

minus 32768 le 119909119894le 32768

(10)



minus15

minus15

minus10

minus10

minus5 minus500

5

5

10

10

5

10

15

20

25

5

75

10

125

15

175

20

119884

119883



minus15

minus1minus1

minus05minus05

00

0505

11

1515

200000400000

600000

800000

10

100

1000

10000

100000

119884

119883


1 1199092le 2 cube










0 10 20 30 40 50 600

5

10

15

20

25

30

35

Runs

Opt

imal

val

ue

Optimiser progress

Run optima


















































119894=2

(119894 minus 1) (11)


















6 Conclusions



0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra











1199091015840


where 1199091015840























1199091015840

119894larr997888 1199091015840

119894+ [max (HM119894) minus 119909

1015840

119894] sdot 119903

1199091015840

119894larr997888 1199091015840

119894minus [1199091015840

119894minus min (HM119894)] sdot 119903

(3)








1199091015840








FW (119894) =


MI2119894 if 119894 lt

MI2

FWmin otherwise(5)



221

100minus1

minus1minus2

minus2

minus3 minus3

119884

119883

2000400060008000

100001200014000

1

10

100

1000

10000





119891 (x) =

119899minus1

sum

119894=1

100 (119909119894+1

minus 1199092

119894) + (1 minus 119909

119894)2

minus 2048 le 119909119894le 2048

(6)






119891 (x) = 10119899 +

119899

sum

119894=1

(1199092

119894minus 10 cos (2120587119909

119894)) minus 512 le 119909

119894le 512

(7)

minus4

minus44

4

minus2minus2

2

2

00 10

10

20

2030

30 4040

5050

6060

70

70

80

80

119884

119883


minus150

minus150

minus100


0

50

50

100

100024681012

15

3

45

6

75

9

105

12

119884

119883

550minus100


0

50

50

100

1010

119884

119883







119891 (x) =

119899

sum

119894=1

1199092

119894

4000minus

119899

prod

119894=1

cos(119909119894

radic119894

) + 1

minus 600 le 119909119894le 600

(8)





119899

119899

sum

119894=1

1199092

119894)

minus exp(1

119899

119899

sum

119894=1

cos (2120587119909119894)) + 20 + 119890

minus 32768 le 119909119894le 32768

(9)


1 1199092


119891 (x) = (1 + (1199091+ 1199092+ 1)2

times (19 minus 141199091+ 31199092

1minus 14119909

2+ 611990911199092+ 31199092

2))

times (30 + (21199091minus 31199092)2

times (18 minus 321199091+ 12119909

2

1+ 48119909

2minus 36119909

11199092+ 27119909

2

2))

minus 32768 le 119909119894le 32768

(10)



minus15

minus15

minus10

minus10

minus5 minus500

5

5

10

10

5

10

15

20

25

5

75

10

125

15

175

20

119884

119883



minus15

minus1minus1

minus05minus05

00

0505

11

1515

200000400000

600000

800000

10

100

1000

10000

100000

119884

119883


1 1199092le 2 cube










0 10 20 30 40 50 600

5

10

15

20

25

30

35

Runs

Opt

imal

val

ue

Optimiser progress

Run optima


















































119894=2

(119894 minus 1) (11)


















6 Conclusions



0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra














1199091015840








FW (119894) =


MI2119894 if 119894 lt

MI2

FWmin otherwise(5)



221

100minus1

minus1minus2

minus2

minus3 minus3

119884

119883

2000400060008000

100001200014000

1

10

100

1000

10000





119891 (x) =

119899minus1

sum

119894=1

100 (119909119894+1

minus 1199092

119894) + (1 minus 119909

119894)2

minus 2048 le 119909119894le 2048

(6)






119891 (x) = 10119899 +

119899

sum

119894=1

(1199092

119894minus 10 cos (2120587119909

119894)) minus 512 le 119909

119894le 512

(7)

minus4

minus44

4

minus2minus2

2

2

00 10

10

20

2030

30 4040

5050

6060

70

70

80

80

119884

119883


minus150

minus150

minus100


0

50

50

100

100024681012

15

3

45

6

75

9

105

12

119884

119883

550minus100


0

50

50

100

1010

119884

119883







119891 (x) =

119899

sum

119894=1

1199092

119894

4000minus

119899

prod

119894=1

cos(119909119894

radic119894

) + 1

minus 600 le 119909119894le 600

(8)





119899

119899

sum

119894=1

1199092

119894)

minus exp(1

119899

119899

sum

119894=1

cos (2120587119909119894)) + 20 + 119890

minus 32768 le 119909119894le 32768

(9)


1 1199092


119891 (x) = (1 + (1199091+ 1199092+ 1)2

times (19 minus 141199091+ 31199092

1minus 14119909

2+ 611990911199092+ 31199092

2))

times (30 + (21199091minus 31199092)2

times (18 minus 321199091+ 12119909

2

1+ 48119909

2minus 36119909

11199092+ 27119909

2

2))

minus 32768 le 119909119894le 32768

(10)



minus15

minus15

minus10

minus10

minus5 minus500

5

5

10

10

5

10

15

20

25

5

75

10

125

15

175

20

119884

119883



minus15

minus1minus1

minus05minus05

00

0505

11

1515

200000400000

600000

800000

10

100

1000

10000

100000

119884

119883


1 1199092le 2 cube










0 10 20 30 40 50 600

5

10

15

20

25

30

35

Runs

Opt

imal

val

ue

Optimiser progress

Run optima


















































119894=2

(119894 minus 1) (11)


















6 Conclusions



0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra










221

100minus1

minus1minus2

minus2

minus3 minus3

119884

119883

2000400060008000

100001200014000

1

10

100

1000

10000





119891 (x) =

119899minus1

sum

119894=1

100 (119909119894+1

minus 1199092

119894) + (1 minus 119909

119894)2

minus 2048 le 119909119894le 2048

(6)






119891 (x) = 10119899 +

119899

sum

119894=1

(1199092

119894minus 10 cos (2120587119909

119894)) minus 512 le 119909

119894le 512

(7)

minus4

minus44

4

minus2minus2

2

2

00 10

10

20

2030

30 4040

5050

6060

70

70

80

80

119884

119883


minus150

minus150

minus100


0

50

50

100

100024681012

15

3

45

6

75

9

105

12

119884

119883

550minus100


0

50

50

100

1010

119884

119883







119891 (x) =

119899

sum

119894=1

1199092

119894

4000minus

119899

prod

119894=1

cos(119909119894

radic119894

) + 1

minus 600 le 119909119894le 600

(8)





119899

119899

sum

119894=1

1199092

119894)

minus exp(1

119899

119899

sum

119894=1

cos (2120587119909119894)) + 20 + 119890

minus 32768 le 119909119894le 32768

(9)


1 1199092


119891 (x) = (1 + (1199091+ 1199092+ 1)2

times (19 minus 141199091+ 31199092

1minus 14119909

2+ 611990911199092+ 31199092

2))

times (30 + (21199091minus 31199092)2

times (18 minus 321199091+ 12119909

2

1+ 48119909

2minus 36119909

11199092+ 27119909

2

2))

minus 32768 le 119909119894le 32768

(10)



minus15

minus15

minus10

minus10

minus5 minus500

5

5

10

10

5

10

15

20

25

5

75

10

125

15

175

20

119884

119883



minus15

minus1minus1

minus05minus05

00

0505

11

1515

200000400000

600000

800000

10

100

1000

10000

100000

119884

119883


1 1199092le 2 cube










0 10 20 30 40 50 600

5

10

15

20

25

30

35

Runs

Opt

imal

val

ue

Optimiser progress

Run optima


















































119894=2

(119894 minus 1) (11)


















6 Conclusions



0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra











119891 (x) =

119899

sum

119894=1

1199092

119894

4000minus

119899

prod

119894=1

cos(119909119894

radic119894

) + 1

minus 600 le 119909119894le 600

(8)





119899

119899

sum

119894=1

1199092

119894)

minus exp(1

119899

119899

sum

119894=1

cos (2120587119909119894)) + 20 + 119890

minus 32768 le 119909119894le 32768

(9)


1 1199092


119891 (x) = (1 + (1199091+ 1199092+ 1)2

times (19 minus 141199091+ 31199092

1minus 14119909

2+ 611990911199092+ 31199092

2))

times (30 + (21199091minus 31199092)2

times (18 minus 321199091+ 12119909

2

1+ 48119909

2minus 36119909

11199092+ 27119909

2

2))

minus 32768 le 119909119894le 32768

(10)



minus15

minus15

minus10

minus10

minus5 minus500

5

5

10

10

5

10

15

20

25

5

75

10

125

15

175

20

119884

119883



minus15

minus1minus1

minus05minus05

00

0505

11

1515

200000400000

600000

800000

10

100

1000

10000

100000

119884

119883


1 1199092le 2 cube










0 10 20 30 40 50 600

5

10

15

20

25

30

35

Runs

Opt

imal

val

ue

Optimiser progress

Run optima


















































119894=2

(119894 minus 1) (11)


















6 Conclusions



0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra
















0 10 20 30 40 50 600

5

10

15

20

25

30

35

Runs

Opt

imal

val

ue

Optimiser progress

Run optima


















































119894=2

(119894 minus 1) (11)


















6 Conclusions



0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra




















































119894=2

(119894 minus 1) (11)


















6 Conclusions



0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra


































119894=2

(119894 minus 1) (11)


















6 Conclusions



0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra




















119894=2

(119894 minus 1) (11)


















6 Conclusions



0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra





















6 Conclusions



0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra










0 100000 200000 300000 400000 5000000

5

10

15

20

25

Fitn

ess v

alue

Optimiser progress



Iterations






Acknowledgment


References




































Volume 2014




Journal of











Journal of


Function Spaces






Algebra
































Volume 2014




Journal of











Journal of


Function Spaces






Algebra
















Volume 2014




Journal of











Journal of


Function Spaces






Algebra









Documents

Research Article Generalised Adaptive Harmony Search: A