Evolving Search Spaces to Emphasize the Performance Differenceof Real-Coded Crossovers Using Genetic Programming

Shinichi Shirakawa, Member, IEEE, Noriko Yata, and Tomoharu Nagao, Member, IEEE

Abstract— When we evaluate the search performance of anevolutionary computation (EC) technique, we usually apply itto typical benchmark functions and evaluate its performancein comparison to other techniques. In experiments on limitedbenchmark functions, it can be difficult to understand thefeatures of each technique. In this paper, the search spacesthat emphasize the performance difference of EC techniques areevolved by Cartesian genetic programming. We focus on a real-coded genetic algorithm, which is a type of genetic algorithmthat has a real-valued vector as a chromosome. In particular,we generate search spaces using the performance differenceof real-coded crossovers. In the experiments, we evolve thesearch spaces using the combination of three types of real-coded crossovers. As a result of our experiments, the searchspaces that exhibit the largest performance difference of twocrossovers are generated for all the combinations.

I. INTRODUCTION

Numerous evolutionary computation (EC) techniques andrelated improvements, showing their effectiveness in variousproblem domains, have been proposed. Especially, variousapproaches have attempted to solve function-optimizationproblems that search minimum or maximum values of real-valued function. Typical techniques to solve EC function-optimization problems include real-coded genetic algorithms(RCGAs) [1], [2], [3], [4], [5], a covariance matrix adaptationevolution strategy (CMA-ES) [6], particle swarm optimiza-tion (PSO) [7], and differential evolution (DE) [8]. Whenwe evaluate the search performance of an EC technique,we usually apply the techniques to the typical benchmarkfunctions and evaluate the performance of the techniques incomparison to other techniques. When there are only slightdifferences in the performance of the techniques in regard tobenchmark functions, it is difficult to understand the uniquefeatures of each technique. In addition, in this case, theperformance relative to other benchmark functions and real-world problems is not clear. Research exists that theoreticallyanalyzes the behavior of EC techniques. However, it isdifficult to analyze and understand the theory of complexEC techniques.

In this situation, there are several research studies that gen-erate a search space that increases the performance differenceof two EC techniques. Oltean succeeded in evolving one-dimensional real-valued search problems and binary-valuedsearch problems with a large performance difference of twosimple (1 + 1) ES in which settings are different [9]. In [10],

The authors are with the Graduate School of Environment and InformationSciences, Yokohama National University, 79-7, Tokiwadai, Hodogaya-ku, Yokohama, Kanagawa 240-8501, Japan (phone: +81 45 3394136;email: shinichi.shirakawa@gmail.com, yata@nlab.sogo1.ynu.ac.jp,nagao@ynu.ac.jp).

[11], Langdon and Poli evolved new problems that maximizethe difference in performance between two optimizers (e.g.,PSO, DE, CMA-ES, and hill climbing) using genetic pro-gramming (GP) [12], [13]. In this research, the performancedifference between the two optimizers is assumed to be afitness function of GP, and search spaces represented bycombining the operators {+,−,×,÷} are evolved. As aresult, they have generated the search problems to which acertain optimizer is superior (or inferior) to other optimizers.This approach to generate search problems is useful for theanalysis and understanding of search techniques.

From the standpoint of evolving problems, Hemert evolvedcombinatorial problem instances that are difficult to solve[14]. The technique is applied to three combinatorial op-timizations: binary constraint satisfaction, Boolean satisfia-bility, and the travelling salesman problem. This techniquesuccessfully evolved difficult problems.

In this paper, we focus on a real-coded genetic algorithm(RCGA), which is a type of GA that has a real-valued vectoras a chromosome. In particular, we generate search spacesusing the performance differences of real-coded crossovers.Various real-coded crossovers have been proposed that haveunique characteristics. In our experiments, we use Cartesiangenetic programming (CGP) [15], [16], which is a graph-based GP technique. In addition, we extend the objectivefunctions to two of the performance differences and a numberof active nodes in CGP and attempt to generate multiplesearch spaces with an evolution using a multiobjective evo-lutionary algorithm (MOEA). Finally, we generate the searchspaces to which the random search performs better thanRCGA. We expect that our method will help to analyze andunderstand the influence on a search of the different searchalgorithms (in this paper, the differences between crossovermethods) on the search, and we expect to provide usefulfeedback for the development of EC methods.

An overview of our method is presented in the nextsection. In Sections III to V, we show the results of threetypes of experiments. Finally, in Section VI, we present ourconclusions and possible future work.

II. GENERATION OF SEARCH SPACES TO EMPHASIZE THE

PERFORMANCE DIFFERENCES OF REAL-CODED

CROSSOVERS

In this paper, we evolve search spaces that emphasizethe performance difference between real-coded crossoversbased on references [10], [11]. Fig. 1 illustrates the outlineof our method. Each individual GP program corresponds toa function (expression) that represents a search space. The

RCGA 1

RCGA 2

-

Performance 1

Performance 2

⊿Performance

Generate search space with higher

⊿performance using CGP

xy fitness

Search space represented by CGP

Fig. 1. Outline of our method that generates the search spaces to emphasizethe performance difference of real-coded crossovers.

performance difference of two RCGAs that differ in termsof crossover method is assumed to be a fitness function ofGP in the experiment, and the search space that increases theperformance difference is evolved. We expect to analyze thefeatures that each real-coded crossover has by generating thesearch spaces with GP. In the experiment, two-dimensionalsearch spaces are generated, because the grasp of the land-scape of the search space is easy. Hence, each individualGP program corresponds to a function that returns a certainfitness value f from two input values (x, y).

In this paper, we use CGP [15], [16], in particular, aprogram that represents the network structure as a GP model.CGP constructs feed-forward network-structured programs.The nodes of CGP are categorized into three types: inputnodes, arithmetic-operation nodes, and output nodes. Inputnodes correspond to input values, arithmetic-operation nodesexecute arithmetic operations, and the output values of theprogram are obtained from the output nodes. One of thebenefits of this type of representation is that it allows theimplicit reuse of nodes in its network. To adopt an evo-lutionary method, CGP uses genotype-phenotype mapping.The feed-forward network structure is encoded in the formof a linear string. Numbers are allocated to each node inadvance. Increasingly large numbers are allocated to inputnodes, arithmetic operation nodes, and output nodes. Thenodes take their input from the output of the previous nodesin a feed-forward manner. The genotype in CGP has a fixed-length representation and consists of a string that encodesthe node function ID and connections of each node in thenetwork. However, the number of nodes in the phenotypecan vary, as not all the nodes encoded in the genotype mustbe connected. This allows the existence of inactive nodes.Only the mutation is usually used as a genetic operator ofCGP. The mutation operator affects an individual as follows:

• Randomly select several genes according to the muta-tion rate Pm for each gene.

• Randomly change the selected genes under the con-straints of the structure.

The (1 + 4) evolution strategy ((1 + 4) ES) is adopted asthe generation-alternation model. Because CGP has inactive

TABLE I

ARITHMETIC OPERATION NODES OF CGP USED IN THE EXPERIMENTS

OF GENERATING THE SEARCH SPACES.

Mark # Args Definition

+ 2 x1 + x2

− 2 x1 − x2

× 2 x1 × x2

÷ 2 x1 ÷ x2

(if x2 = 0.0 return x1)√| | 1√|x1|

sin 1 sin(x1)cos 1 cos(x1)0.0 0 Constant value 0.01.0 0 Constant value 1.010.0 0 Constant value 10.0−1.0 0 Constant value −1.0

π 0 Constant value π

nodes, a neutral effect on fitness is caused by the geneticoperation (called neutrality [17]). In the (1 + 4) ES, theoffspring is selected if it exhibits the same best fitness asthe parent, then the searching point moves even if the fitnessis not improved. Therefore, we believe that efficient searchis achieved through a simple generation-alternation model.Many genetic operations in which there is no change in activenodes occur because they use a small mutation probability.In addition, the fitness calculation can be omitted when thegenetic operations occur only on inactive nodes.

Table I shows the arithmetic operation nodes of CGP usedin the experiments. CGP is expected to represent a complexsearch space using a combination of these nodes.

The fitness assignment of each individual is based onthe results of the searches by two RCGAs for the searchspace represented by CGP. In this manner, a search space isobtained in which the performance difference of two RCGAsincreases. The fitness function of each individual in CGP isdefined in the following equation:

Fcgp =1N

N∑n=1

T∑t=1

log(

g1(t, n)g2(t, n)

),

gi(t, n) =

⎧⎪⎨⎪⎩

fmin if fi(t, n) < fmin

fmax if fi(t, n) > fmax

fi(t, n) otherwize

(1)

where N is the number of trials of each RCGA, T is themaximum number of generations in RCGA, and fi(t, n) isthe best evaluation value on the tth generations of the nthtrials. The value of gi(t, n) is limited within the range of[fmin, fmax], and g1 and g2 are the values obtained fromthe searches of RCGA 1 and 2, which are the differentsettings, respectively. In the experiments, we set N = 10,T = 500, fmin = 10−8, fmax = 10.0, respectively. The fitnessfunction Fcgp is an accumulation of the differences of the logvalues of the best evaluation values of two RCGAs obtainedin each generation. Therefore, the process of the search isalso included in the performance assessment (obtaining agood evaluation value earlier in the search shows that the

TABLE II

CGP PARAMETERS USED IN THE EXPERIMENTS.

Parameter Value

Number of generations 15000Mutation rate Pm 0.02Number of arithmetic operation nodes 200Number of input nodes 2Number of output nodes 1

performance is high). In the experiments for this paper,RCGAs search for the minimum value in the search spacesrepresented by CGP. Thus, smaller the value of gi(t, n),higher is the performance of the RCGA. Larger the value ofFcgp, greater is the performance difference of the RCGAs inthat search space. It is shown that the performance of RCGA2 is higher than that of RCGA 1 in the search space with alarge value for Fcgp. A search space with a large differencebetween the performance of two RCGAs is evolved withCGP using such a fitness function.

Here we discuss the difference between the present studyand the studies of [10], [11]. The search spaces are evolvedusing the performance difference between PSO, DE, andCMA-ES by tree-based GP in [10], [11]. In our study,we focus on real-coded crossovers in RCGA and evolvethe search spaces using the performance difference betweenthem using CGP. While the node functions of GP are only{+,−,×,÷} in [10], [11], we have detailed more nodefunctions, as shown in Table I. Therefore, we expect togenerate more complex search spaces. Our approach alsodiffers in such a way that our fitness function considers theprocess of the searches. In section IV, the number of nodesin CGP is added to the objective function, and we attempt toobtain not only one search space but also a variety of searchspaces simultaneously by multiobjective optimization (this isalso a difference from previous studies). We can then observewhich elements increase the performance difference.

III. EXPERIMENTS OF GENERATING THE SEARCH SPACES

A. Experimental Settings

The CGP parameters used in the experiments are listedin Table II. The search space is generated using the perfor-mance difference of two RCGAs with different real-codedcrossovers. We use three real-coded crossovers: BLX-α [1],unimodal normal-distribution crossover (UNDX) [2], [3], andsimplex crossover (SPX) [4]. The search space is generatedon the basis of combination of these crossover methods inthe experiments.

BLX-α uniformly and independently picks new individ-uals with values that lie in [di − αdi, di + αdi] for eachparameter, where di is the distance between the parents. Theequation of BLX-α is as follows:

ci = u(min(p1i, p2i) − αI,max(p1i, p2i) + αI) (2)

I = |p1i − p2i| (3)

where ci is the ith parameter of offspring, p1i and p2i are theith parameter of parents 1 and 2, respectively, and u(m,n)is a uniformly distributed random number among [m,n]. Weset the parameter α = 0.366 in the experiments.

UNDX deals with variable dependence (non-separability),although the performance of BLX-α deteriorates in thesearch space with variable dependence. UNDX generatesoffspring using normally distributed random numbers definedby three parents. Offspring are generated around the linesegment connecting the two parents: parent 1 and parent 2.The third parent, parent 3, is used to decide the standarddeviation of the distance to the axis connecting parents 1and 2. The generation of offspring �C is as follows:

�C = �m + ξ �d + D

n−1∑i=1

ηi�ei (4)

where,

�m = (�P1 + �P2)/2, �d = �P2 − �P1, (5)

D =

√|�P3 − �P1|2 − [�d · (�P3 − �P1)]2

|�d|2, (6)

ξ ∼ N(0, σ2ξ ), ηi ∼ N(0, σ2

η), (7)

�P1, �P2 and �P3 are parent vectors of parents 1, 2, and3, respectively, �ei is the orthogonal basis vector spanningthe subspace perpendicular to the line connecting parents 1and 2, D is the distance of the third parent from the lineconnecting parents 1 and 2, n is the number of dimensionsin the search space, and N(0, σ2) is a normally distributedrandom number. We set the parameters σξ = 0.5, ση =0.35/

√n, which are the recommended values [2], [3].

SPX is a multiparent recombination operator. It generatesoffspring vector values by uniformly sampling values fromthe simplex formed by n+1 (n is the number of dimensionsin the search space) parent vectors. SPX is independentfrom coordinate systems in generating offspring. The SPXprocedure is as follows:

1) Calculate the center of mass �G from the parents�P1, ..., �Pn+1.

�G =1

n + 1

n+1∑i=1

�Pi (8)

2) Calculate �xk and �Ck for k = 1, ..., n + 1.

�xk = �G + ε(�Pk − �G) (k = 1, ..., n + 1) (9)

�Ck =

⎧⎪⎨⎪⎩

�0 (k = 1)rk(�xk−1 − �xk + �Ck−1)(k = 2, ..., n + 1)

(10)

where ε is the positive-valued parameter called expan-sion rate. rk is the random number calculated by thefollowing equation:

rk = (u(0, 1))1k (k = 2, ..., n + 1) (11)

where u(0, 1) is a uniformly distributed random num-ber among [0, 1].

Fig. 2. Conceptual illustration of the distribution of the offspring generatedfor each real-coded crossover (two dimensions). BLX-α and SPX generatethe offspring using uniformly distributed random numbers, while UNDXgenerates the offspring using normally distributed random numbers.

3) Generate the offspring �C.

�C = �xn+1 + �Cn+1 (12)

We set the parameter ε =√

n + 2(= 2.0) that is therecomended value [4].

Fig. 2 illustrates the distribution of the offspring gener-ated for each real-coded crossover in the case of a two-dimensional search space.

We use the minimal generation gap (MGG) model [18]as a generation-alternation model. It is a steady-state modelproposed by Satoh et al., which exhibits the desirableconvergence property of maintaining the diversity of thepopulation, and is widely used in RCGAs. The MGG modelis summarized as follows:

1) Set the generation counter t = 0. Randomly generatethe individuals as the initial population P (t).

2) Select a set of μ parents for the crossover by randomsample from the population, where μ denotes thenumber of parents used for the crossover.

3) Generate a set of nc children by applying the crossoverto the parents.

4) Select two individuals from the set of two parents (themain two parents in UNDX and the randomly selectedtwo parents in SPX) and nc children. One is the elitistindividual and the other is the individual from a rank-based roulette-wheel selection. Then replace the twoparents with the new two individuals in populationP (t) to get population P (t + 1).

5) Stop if a certain specified condition is satisfied. Oth-erwise, set t = t + 1, and go to step 2.

The MGG model localizes its selection pressure, not to thewhole population (as simple GA or steady state does), ratheronly to the family (offspring and parents).

Common RCGA parameters are used in each real-codedcrossover. The population size 30, children size nc = 20,and the maximum number of generations T = 500 areused in the experiments. We confirm from the exploratoryexperiment that we can perform adequate searches usingthese parameters, because the search space represented byCGP is two-dimensional. The range of the search spaces isassumed to be [−10, 10], and initial individuals are gener-ated with uniformly distributed random numbers within thisrange. When an offspring is generated outside the range by

TABLE III

AVERAGE OF THE FITNESS VALUES OF THE SEARCH SPACES OBTAINED

BY EACH SETTING (AVERAGE OVER 10 TRIALS WITH THE SAME

PARAMETER SET). THE VALUES SHOWN IN PARENTHESES ARE

CALCULATED USING STANDARD DEVIATION.

LoseBLX-α UNDX SPX

BLX-α – 4778 5463– (1533) (1473)

Win UNDX 7521 – 5911(1918) – (1284)

SPX 7411 2854 –(2024) (1244) –

crossover, its parameters are corrected on the boundary. Itbreaks off the search for the RCGA when an evaluation valuethat is smaller than fmin is obtained during the search. Whensearching by each RCGA, the upper bound is not used in thefitness assignment in RCGA, although fmax is set to calculatethe fitness value of CGP with Equation ((1)). In addition,when searching by each RCGA, the random numbers foreach search of RCGAs are initialized using the same seeds.Therefore, the same fitness evaluation can be given eachtime for the same search space represented by CGP. Weexperiment ten different runs with the same parameter setat each combination of each real-coded crossover.

B. Experimental Results

We evolve the search spaces 10 times by each setting. Theaverage of the fitness values of the search spaces obtainedby each setting is shown in Table III. The numerical valuesin the table show the fitness values when the search spaceis evolved for which the row’s crossover method performsbetter than the column’s crossover method. The values shownin parentheses are calculated using standard deviation. Fromthis table, the search spaces that have the largest performancedifference between two crossovers can be generated for allcombinations.

Fig. 3 shows the landscapes with the highest fitness valueamong the search spaces generated in the combination ofeach real-coded crossover over the course of ten trials. Thehorizontal axis of the search space corresponds to the param-eter x and the vertical axis corresponds to the parameter y.The colors used in the figures indicate the evaluation valueof each coordinate. Yellow indicates high values, and blueindicates low values. We apply each real-coded crossover toeach search space in Fig. 3. Table IV shows the number ofsuccessful runs for each search space over the course of 30runs. We consider the run to be a success when an evaluationvalue that is smaller than fmin is obtained during the search.

Figs. 3 (a) and (b) illustrate the search spaces to whichBLX-α performs better than UNDX and SPX, respectively.Both search spaces are multimodal, and the evaluation valuesare decided almost exclusively on the parameter of x. In aword, there is no variable dependence. The search space inFig. 3 (a) has a minimum value around x = 0, and (b) has a

(a) BLX-α performs better than UNDX (Fitness: 7116) (b) BLX-α performs better than SPX (Fitness: 7669)

(c) UNDX performs better than BLX-α (Fitness: 9540) (d) UNDX performs better than SPX (Fitness: 7854)

(e) SPX performs better than BLX-α (Fitness: 10132) (f) SPX performs better than UNDX (Fitness: 5389)

Fig. 3. Examples of the generated search spaces that maximize the performance difference of real-coded crossovers. The horizontal axis of the searchspace corresponds to the parameter x, and the vertical axis corresponds to the parameter y. The colors in the figures indicate the evaluation value of thecoordinate. Yellow indicates high values, and blue indicates low values.

minimum value near the coordinates, where x is a multiple ofπ. The search spaces without variable dependence (such asFigs. 3 (a) and (b)) are generated, because the performanceof BLX-α deteriorates in the search space with variabledependence. From Table IV, BLX-α discovers the minimumvalue by all the trials, whereas the number of successful runsis a little when UNDX and SPX are applied to the searchspaces.

Fig. 3 (c) illustrates the search space to which UNDX per-forms better than BLX-α. It is necessary to change both the xand y variables to considerably improve the evaluation value

in this search space. In a word, there is variable dependence.Although this search space has minimum values near x =−10, it has a large value with x = −10. Therefore, even if theindividual is generated outside the boundary and the value ofx = −10 is obtained, a minimum value cannot be obtained.Such a situation often appears in real-world problems. Itis necessary to generate the individual near x = −10 toobtain a minimum value in this search space. This searchspace is advantageous for UNDX, because UNDX generatesindividuals with normally distributed random numbers andcan deal with variable dependence. We analyzed the process

TABLE IV

NUMBER OF SUCCESSFUL RUNS OVER THE COURSE OF 30 RUNS WHEN

EACH REAL-CODED CROSSOVER IS APPLIED TO EACH SEARCH SPACE IN

FIG. 3.

BLX-α UNDX SPXFig.3 (a) 30 11 9Fig.3 (b) 30 17 20Fig.3 (c) 16 27 15Fig.3 (d) 21 30 26Fig.3 (e) 14 28 24Fig.3 (f) 25 19 24

of the search by BLX-α, and confirmed that many individualsare generated on the boundary of x = −10. Thus, BLX-αfails in the search and cannot obtain a minimum value. FromTable IV, the number of successful runs is the highest whenUNDX are applied to this search space.

Fig. 3 (d) illustrates the search space to which UNDXperforms better than SPX. Although this search space is mul-timodal and has a minimum value near (x, y) = (−10, 10),it has a large value with y = 10. Therefore, it is necessaryto generate the individual near (x, y) = (−10, 10) to obtaina minimum value in this search space. This search space isalso advantageous for UNDX as well as Fig. 3 (c), becauseUNDX generates individuals with normally distributed ran-dom numbers.

Fig. 3 (e) illustrates the search space to which SPXperforms better than BLX-α. The range of each parameterin Fig. 3 (e) is [9, 10], because the evaluation values increaseconsiderably when parting from the straight line x = y. Thissearch space has variable dependence and has the minimumvalues in x = y. From Table IV, the number of successfulruns of SPX is more than that of BLX-α. However, thenumber of successful runs of UNDX is the highest becausethe performance of UNDX is not considered when the searchspace is evolved.

Fig. 3 (f) illustrates the search space to which SPXperforms better than UNDX. This search space has minimumvalues in the area near y = 0. On the other hand, the areaof large values exists in its neighborhood. The evaluationvalues become small increasing of the parameter of y as theentire landscape of the search space. Although the number ofsuccessful runs of SPX is more than that of UNDX in TableIV, the performance difference is not great. In addition, thenumber of successful runs of BLX-α is the highest becausethe performance of BLX-α is not considered when the searchspace is evolved.

Next, we discuss the functions of the generated searchspaces. The function of the search space in Fig. 3 (b) is asfollows:

f(x, y) =(sin

√| sin(x)| − 10

)(9 + sin(sin(10))) sin (sin(sin(10))) cos(y)

+10√

| sin(x)| (10 + cos(x)) + 10− cos (10(10 + cos(x))) (13)

The function of the search space in Fig. 3 (c) is as follows:

f(x, y) = 20π − 8 +cos(x)

y+

√|10 + y| − y (14)

The functions generated by CGP include a complex onewith many elements and a simple one. Because the functionsthat maximize Equation (1) are generated in this experiment,functions that are difficult to understand are occasionallygenerated. In the next section, the number of nodes in CGPis added to the objective function, and we attempt to obtain avariety of search spaces simultaneously using multiobjectiveoptimization.

IV. GENERATION OF SEARCH SPACES USING A

MULTIOBJECTIVE EVOLUTIONARY ALGORITHM

Only Equation (1) is assumed to be an objective functionfor generating the search spaces in the previous experiments.Therefore, complex functions (search spaces) tend to begenerated. Understanding and analyzing the features of thesearch spaces becomes difficult, because the search spacesto which the performance difference increases tend to becomplex. In this section, we extend the problem to a binaryobjective problem by adding the number of active nodes tothe fitness function of CGP. We also attempt to obtain notonly one search space but also a variety of search spacessimultaneously by multiobjective optimization. We can thenobserve which types of elements expand the performance dif-ference. Equation (1) is used for the first objective function,and the number of active nodes in CGP is used for the secondobjective function. Equation (1) is expected to be maximized,and the number of active nodes are expected to be minimized.We use strength Pareto evolutionary algorithm 2 (SPEA2)[19] as the MOEA technique. SPEA2 assigns fitness valuesto individuals according to dominance and density criteria.The selection of individuals is also based on these criteria.It maintains two populations: a current population and anarchive population. In each generation, the nondominatedsolutions in the current population and the archive are copiedinto the archive of the next generation. Selection occurs fromboth the population and the archive. The parameters of CGPare shown in Table II and the parameters of RCGAs thatare explained in section III-A are used. We use populationsize 5 and archive size 20 as the parameters of SPEA2. Weuse BLX-α and UNDX as the real-coded crossovers in thisexperiment, and the search spaces to which UNDX performsbetter than BLX-α, and to which BLX-α performs betterthan UNDX, are evolved.

The functions of the nondominated solutions obtained bythe MOEA are shown in Tables V and VI. We confirm thatthe performance difference increases as the number of activenodes increases, and there are common components in eachfunction. Multimodality increases the performance differenceof BLX-α and UNDX in Table V. From VI, the variabledependence and the sharp landscape (square root) increasethe performance difference of UNDX and BLX-α. Fig. 4shows the evolved landscapes to which BLX-α performsbetter than UNDX when the number of active nodes is 7, 9,

f =√

|y(y − √|x|)| f = 100√| sin(y)| − cos(y) f = 100

√| sin(y)| − cos(x sin(x))

Fig. 4. Examples of the generated search spaces by the MOEA. The number of active nodes is 7, 9, and 12, respectively. BLX-α performs better thanUNDX in these search spaces.

f = |y − x|0.25 f = |x(y − x)|0.25 f = |(y − x)(cos(x2) + x2)|0.25

Fig. 5. Examples of the generated search spaces by the MOEA. The number of active nodes is 6, 7, and 10, respectively. UNDX performs better thanBLX-α in these search spaces.

TABLE V

FUNCTIONS OF THE NONDOMINATED SOLUTIONS OBTAINED BY THE

MOEA. BLX-α PERFORMS BETTER THAN UNDX IN THESE SEARCH

SPACES.

Function # nodes Fitness√|y| 3 1570√|2y| 4 1593√|xy| 5 1704√|y(y + cos(y))| 6 1775√|y(y − √|x|)| 7 2122

100√| sin(y)| − cos(y) 9 4272

100√| cos(y)| − cos(y) − cos(

√| cos(y)|) 10 4750100

√| cos(y)| − cos(y) − cos(x) 11 4940100

√| sin(y)| − cos(x sin(x)) 12 5782

and 12, respectively. These search spaces have little variabledependence and are the multimodal landscapes. Fig. 5 showsthe evolved landscapes to which UNDX performs better thanBLX-α when the number of active nodes is 6, 7, and 10,respectively. These search spaces have the minimum valuesin x = y, and they have variable dependence. The area wherethe evaluation values are low is formed around x = 0 as thenode increases.

V. GENERATION OF SEARCH SPACES TO EMPHASIZE

PERFORMANCE DIFFERENCE WITH RANDOM SEARCH

In this section, we attempt the evolution of search spaces towhich the random generation of individuals (random search)

TABLE VI

FUNCTIONS OF THE NONDOMINATED SOLUTIONS OBTAINED BY THE

MOEA. UNDX PERFORMS BETTER THAN BLX-α IN THESE SEARCH

SPACES.

Function # nodes Fitnessf = 10 − y 4 37f =

√|y − x| 5 2286f = |y − x|0.25 6 2640f = |x(y − x)|0.25 7 4268f =

√|(y − x)(cos(x2) + x2)| 9 6092f = |(y − x)(cos(x2) + x2)|0.25 10 7144

performs better than real-coded crossovers. The setting of theexperiment is the same as that in section III-A.

As a result, the search spaces to which the random searchperforms better than each real-coded crossover could begenerated. Fig. 6 shows the example of the search space towhich the random search performs better than BLX-α. Thefitness value of this search space calculated by Equation (1)is 9901. The favorable evaluation values are macroscopicallyobtained when the value of x is small in this search space.However, the areas that have a minimum value actually existaround the value of x from 9 to 10. This search space belongsto a so-called deceptive problem. BLX-α fails to find theglobal optimum for such a deceptive problem, because thepopulation converges to coordinates with small values ofx. In this situation, the probability that the random search

Fig. 6. Example of the search space to which the random search performsbetter than BLX-α.

discovers the optimum solution compared to RCGAs is high.The search spaces to which the random search performsbetter than UNDX and SPX have the tendency to a deceptivestructure similar to Fig. 6. Although the search space shownin Fig. 6 is a typical problem for which the EC techniquesfail in the search, we think that being able to generate sucha search space by our method is one of the valuable results.

VI. CONCLUSIONS AND FUTURE WORK

In this paper, the search spaces that emphasize the per-formance difference of real-coded crossovers were evolved.We used the combination of three types of real-codedcrossovers in the experiments: BLX-α, UNDX, and SPX.In the experimental results, the search spaces that havethe largest performance difference between two crossoverscould be generated for all the combinations. After that, thenumber of nodes in CGP was added to the objective function,and we evolved a variety of search spaces simultaneouslyusing a multiobjective evolutionary algorithm. Moreover, weconfirmed that our method generates the search spaces towhich the random search performs better than the search ofthe real-coded crossovers.

In future, we plan to evolve the search spaces not only touse the difference of the real-coded crossovers, but also to usethe difference of the generation alternation models and eachparameter. Search spaces can be generated using the per-formance difference between other evolutionary computationmethods and the multiobjective evolutionary algorithms. Fur-thermore, higher-dimensional search spaces will be evolvedthrough the search spaces of the two dimensions that wereevolved in this paper. Finally, we expect to better understandthe features and the behavior of evolutionary computationtechniques from the analysis of the search spaces generatedby our method. We also expect that the results of our method

will contribute to the development of a new evolutionaryalgorithm and the discovery of a new problem domain.

REFERENCES

[1] L. J. Eshelman and J. D. Schaffer, “Real coded genetic algorithms andinterval-schemata,” in Foundations of Genetic Algorithms 2. MorganKaufmann Publishers, 1993, pp. 187–202.

[2] I. Ono and S. Kobayashi, “A real-coded genetic algorithm for functionoptimization using unimodal normal distribution crossover,” in Pro-ceedings of the 7th International Conference on Genetic Algorithms(ICGA ’97). Morgan Kaufmann Publishers, 1997, pp. 246–253.

[3] H. Kita, I. Ono, and S. Kobayashi, “Theoretical analysis of theunimodal normal distribution crossover for real-coded genetic algo-rithms,” in Proceedings of the 1998 IEEE Congress on EvolutionaryComputation (CEC ’98), 1998, pp. 529–534.

[4] S. Tsutsui, M. Yamamura, and T. Higuchi, “Multi-parent recombi-nation with simplex crossover in real coded genetic algorithms,” inProceedings of the Genetic and Evolutionary Computation Conference1999 (GECCO ’99). Morgan Kaufmann Publishers, 1999, pp. 657–664.

[5] K. Deb, A. Anand, and D. Joshi, “A computationally efficient evo-lutionary algorithm for real-parameter optimization,” EvolutionaryComputation, vol. 10, no. 4, pp. 371–395, 2002.

[6] N. Hansen and A. Ostermeier, “Completely derandomized self-adaptation in evolution strategies,” Evolutionary Computation, vol. 9,no. 2, pp. 159–195, 2001.

[7] J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Pro-ceedings of the 1995 IEEE International Conference on Neural Net-works, 1995, pp. 1942–1948.

[8] R. Storn and K. Price, “Differential evolution – a simple and efficientheuristic for global optimization over continuous spaces,” Journal ofGlobal Optimization, vol. 11, no. 4, pp. 341–359, 1997.

[9] M. Oltean, “Searching for a practical evidence of the no free lunch the-orems,” in Biologically Inspired Approaches to Advanced InformationTechnology: First International Workshop, BioADIT 2004, ser. LNCS,vol. 3141. Springer-Verlag, 2004, pp. 472–483.

[10] W. B. Langdon and R. Poli, “Evolving problems to learn aboutparticle swarm and other optimisers,” in Proceedings of the 2005 IEEECongress on Evolutionary Computation (CEC ’05), vol. 1, 2005, pp.81–88.

[11] W. B. Langdon and R. Poli, “Evolving problems to learn about particleswarm optimizers and other search algorithms,” IEEE Transactions onEvolutionary Computation, vol. 11, no. 5, pp. 561–578, 2007.

[12] J. R. Koza, Genetic Programming: On the Programming of Computersby Means of Natural Selection. MIT Press, 1992.

[13] R. Poli, W. B. Langdon, and N. F. McPhee, A Field Guide to GeneticProgramming. Published via http://lulu.com and freely avail-able at http://www.gp-field-guide.org.uk, 2008, (Withcontributions by J. R. Koza).

[14] J. I. van Hemert, “Evolving combinatorial problem instances that aredifficult to solve,” Evolutionary Computation, vol. 14, no. 4, pp. 433–462, 2006.

[15] J. F. Miller and P. Thomson, “Cartesian genetic programming,” inGenetic Programming: 3rd European Conference, EuroGP 2000, ser.LNCS, vol. 1802. Springer-Verlag, 2000, pp. 121–132.

[16] J. F. Miller and S. L. Smith, “Redundancy and computational efficiencyin cartesian genetic programming,” IEEE Transactions on EvolutionaryComputation, vol. 10, no. 2, pp. 167–174, 2006.

[17] T. Yu and J. F. Miller, “Neutrality and the evolvability of booleanfunction landscape,” in Genetic Programming: 4th European Confer-ence, EuroGP 2001, ser. LNCS, vol. 2038. Springer-Verlag, 2001,pp. 204–217.

[18] H. Satoh, M. Yamamura, and S. Kobayashi, “Minimal generationgap model for considering both exploration and exploitations,” inProceedings of the IIZUKA’96, 1996, pp. 494–497.

[19] E. Zitzler, M. Laumanns, and L. Thiele, “SPEA2: Improving theStrength Pareto Evolutionary Algorithm,” Computer Engineering andNetworks Laboratory (TIK), ETH Zurich, Zurich, Switzerland, Tech.Rep. 103, 2001.