[IEEE 2012 International Conference on Machine Learning and Cybernetics (ICMLC) - Xian, Shaanxi, China (2012.07.15-2012.07.17)] 2012 International Conference on Machine Learning and

Proceedings of the 2012 International Conference on Machine Learning and Cybernetics, Xian, 15-17 July, 2012

ART-ENHANCED MODIFIED BINARY DIFFERENTIAL EVOLUTION

ALGORITHM FOR OPTIMIZATION

CHUN-YIN WU1, KUAN-SHIEN nu2

iProfessor, Department of Mechanical Engineering, Tatung University, Taipei, 104, Taiwan 2Graduate Student, Department of Mechanical Engineering, Tatung University, Taipei, 104, Taiwan

E-M AIL: [email protected]@ms4.hinet.net

Abstract: Differential evolution(DE) is a heuristic optimization

method with a relatively simple and efficient form of mutation

and crossover and it has been applied to solve many real world

optimization problems in real-valued search space. Modified

binary differential evolution(MBDE) with a simple binary

mutation mechanism based on a logical operation is suitable

for dealing with binary and continuous optimization problems.

In this study, the modified binary differential evolution is

enhanced by using adaptive resonance theory(ART) to classify

binary image pattern of population into groups for balancing

exploration and exploitation in optimization search. The

diversity and convergence of search are both enhanced by

applying ART clustering strategy. Different types of

optimization problems consisting of test function optimization

and topology optimization of structure are used to illustrate

the high viability of the proposed algorithm in optimization.

Keywords: Modified binary differential evolution; Adaptive

resonance theory; Exploration and exploitation

1. Introduction

Differential evolution (DE), invented by Stron and Price [1,2] in 1995, is one of the evolutionary algorithms used to solve optimization problems in continuous space. It has been recognized as a powerful population-based global optimization method and has been successfully applied in many fields of optimization problems [3-5]. However it is not easy to apply DE to solve binary-based optimization problem. Thus, new strategies for DE algorithms have been proposed to evolve solutions for binary-valued optimization problems. Pampani et al. [6] proposed a binary DE that used the trigonometric function to transfer the design variables from continuous-space into binary-space. Although this method can be used to solve the optimization problems based on binary-space, the process cannot be reversed by transforming binary data back into real value data. Xingshi et al. [7] developed a binary DE based on an

978-1-4673-1487-9/12/$31.00 ©2012 IEEE

artificial immune system using a logical operator. The idea is to transfer subtraction, multiplication and addition of mutation operator to XOR, AND, and OR operators. However, this method has higher probability to product code "1" in the evolution process by using OR operator, it will restrict the diversity in searching the optimum solution.

Evolutionary algorithm is an optimization process comprising two important aspects, exploration and exploitation.[8-11] The exploration search discovers potential offspring in new search regions. But the exploitation search utilizes promising solutions already identified to increase the convergence of search process. If search is too explorative, evolutionary progress may rapidly degenerate into a random search. Conversely, if the degree of exploitation is too high, premature convergence may make the significant areas of the search space remaining under explored. Intelligent balance between these two aspects may drive the search process towards better optimum results and faster convergence rates.

Diversity maintenance is important in multi-modal and multi-objective optimization problems. Clustering techniques are applied in many studies[12-l5] to achieve the correct balance between exploration and exploitation. DE algorithms work well when the population covers the entire design space. But premature may be detected when all the members of the population are in a high attraction area of a local optimum.

This study proposes a clustering-based differential evolution algorithm for keeping balance between exploration and exploitation. The number of local optimum solutions of most optimization problems is not known before search. The clustering technique selected in this study is an unsupervised learning neural network model for clustering binary image, adaptive resonance theory ( ART). The main function of the ART algorithm is to classify population pattems into groups to maintain the diversity of population during search process. The common feature of members of population in same pattem group is used in

1838


mutation operator of MBDE to increase the convergence rates. This paper presents an ART-enhanced modified binary differential evolution ( ART-MBDE) algorithm to tackle parametric and nonparametric optimization problems.

2. Differential evolution

Differential evolution (DE) [1,2] is developed as a population-based algorithm for real-valued optimization problems. The initial population is randomly generated by a uniform random number between the lower and upper bounds of each of the design variables. The objective function values for all individuals are calculated for three main iterative steps of DE, mutation, crossover, and selection.

For each individuals' vectors Xi.G in the population, DE uses a mutation operation to generate new mutated vectors according to the following equation:

Vi,G+ 1 = Xbest,G + F (Xrl,G - Xr2,G), i= 1 ,2,3 ...... N P (1) The index G is used as an index of the current iteration.

In Eq. (1), rl and r2 are generated between 1 and N P. Two random target vectors, Xrl,G and Xr2,G, are chosen in the population of the current generation G. The vectors Xbest,G and Vi,G+ 1 are the best target vector and the mutated vector of the current generation, respectively. Scaling factor F is a real value between zero and one.

In the crossover operator, the trial vector Ui,G+l is generated by a combination of the mutated vector, Vi,G+h and the target vector, Xi,G as shown in Eq. (2). The CR parameter used in Eq. (2) represents the crossover probability, where j is the variable index. If the random number R is smaller than the CR value, or component j is not the same as i, the variable of mutation vector, Vi,G+l ,will be chosen as the variable of the trial vector. Otherwise, the variable of the target vector is selected as the variable of the trial vector.

Ui,G+l = Vi,G+l , if R � Cr or j i- i

Ui,G+l =Xi,G , if R>Cr or j =i (2) After the mutation and crossover operations, all trial

vectors Ui,G+l are identified as candidates for selection operations. The trial vector Ui,G+l is then compared with the target vector Xi,G to select individuals for the next generation. The selection operator is as follows:

Xi,G+l = Ui,G+l , if j(Ui,G+l) > j(Xi,G) Xi,G+l =Xi,G , if j(Ui,G+l) � j(Xi,G) , i=I,2,3 ... N P (3)

2.1. Modified binary differential evolution

Modified binary differential evolution (MBDE) uses binary string representation for each individual within the

population. The initial population of the MBDE is randomly generated for a value of 0 or 1. The three main steps of binary DE - mutation, crossover, and selection -are iterated until the convergence state is satisfied.

Xingshi et al. [7] proposed a logical mutation method for binary mutation mechanisms. The logical operation was used to replace the vector operations of the mutation mechanism in the original differential evolution. The subtraction of two random selected vectors was represented by the logical operation XOR and the multiplication factor F was replaced by the AND operator. The OR operator was applied to replace the operation for the addition of two randomly selected vectors. However, the OR operator used in the binary mutation mechanism proposed by Xingshi et al. [7] had a higher probability of production of bit 1 in the evolution process. This disproportionate probability may restrict the search diversity of the optimum solution.

Another binary mutation mechanism is proposed for MBDE. The binary mutation of MBDE is illustrated in equations (4) and (5). Eq. (4) represents the rand-to-best/lIexp mutation mechanism, while Eq. (5) expresses the best/lIexp mutation mechanism of continuous DE. Equations (4) and (5) describe the new binary mutation approach of common bits and different bits among Xi,G , Xrl,G and Xbest,G individuals. The objective of these equations is to discover search directions for the Xi,G individual corresponding to the relative position of the best or randomly selected individuals in the binary representation

Vi,G+l=Fl(Xi,G EF> xrl,G)+F2 !(Xi,G EF> Xrl,G), i=I,2,3 ... N P (4)

Vi,G+l=F1 (Xbest,G EF> xl,G)+F2 !( Xbest,G EF> Xi,G), i=I,2,3 ... N P (5) The underlying idea regarding this new binary

mutation mechanism is that it finds the pattern of common features between two individuals. In Eq. (4), the difference bits and the common bits between the Xi,G and Xrl,G solutions are determined using the XOR operation. The difference bits of Xi,G are mutated from 0 to 1 or 1 to 0 at mutation rate Fl. Similarly, the common bits of Xi,G are mutated at mutation rate F2. Generally, the mutation rate F 1 for the difference bits is higher than the mutation rate F2 for the common bits, because there is a higher probability that the common bits may become the feature pattern of the final solution.

The binary crossover operator is used to build a trial solution Ui,G+l from the mutated solution Vi,G and the original solution Xi,G' The underlying concept behind the binary crossover mechanism of MBDE is the same as crossover mechanism of the original DE. The difference between MBDE and DE concerns the component data type.

1839


3. ART-enhanced MBDE

Adaptive resonance theory( ART) [16,17] IS an unsupervised learning neural network model and is designed for clustering binary image pattern. The main function of the ART algorithm used in this study is to classify image patterns which are converted from the binary code of real-valued design variable or shape topology of the structure. Stability and plasticity are two key features of the ART algorithm. The stability of a network is the ability to properly retain the character of the system even when some new information is put into it. On the contrary the ability of a network to adapt and learn new pattern well at any stage of operation is call plasticity of the system. The ART algorithm can find the balance between stability and plasticity of the system. So the long term memory of the ART algorithm is used to keep system stable in the classification process and it can also create new group when there is no matched pattern in the classified groups.

The incorporation of a vigilance test allows the ART architecture to resolve the stability-plasticity dilemma. The resonance state will be gotten soon for an input pattern and the stored pattern of the output layer will be updated if the information has been continuously learned by the network. On the contrary, if the information isn't easy to be recognized the network will immediately search through all stored patterns of the output layer for a best match. If there is no matched pattern passed the vigilance test, the input pattern is used to create new a new pattern in the output layer.

The binary-coded image pattern is used for representation of problem dependent conversion mechanism in optimization problems. For parametric optimization, the vector of values of design variables is converted to a binary image pattern as shown in Fig. 1. The topology optimization is used for verification of ART-MBDE in nonparametric optimization and the shape topology is converted to a binary image pattern as shown in Fig. 2.

11 0 1 0 110 1 0

o I 0 I 0 1 0 1

1 1 0 0 1 1 0 0

0 0 1 1 0 0 1 0

1 0 I I 0 I I 0

1 1 1 0 0 0 1 0

0 0 0 1 1 1 0 0

1 O_�,� 1 !��,�

Figure 1 Binary pattern of vector of design variables

1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 1 0 0 1 1 0 0 0

Figure 2 Binary pattern of shape topology of structure

In this study the ART is proposed not only for recognizing and clustering binary image patterns of population but also for diversity maintenance and convergence improvement in ART-MBDE algorithm. The primary purpose of the ART in the algorithm is to classify the population into groups with similar patterns. The diversity of population distribution is controlled by the value of vigilance parameter and it can help prevention of premature in evolution process. On the other hand the ART is also used to determine the neighbors and degree of similarity for each binary patterns of population. The common characteristic of same group pattern is applied in ART-MBDE algorithm to accelerate the convergence in optimization search. The best one with highest objective function value of patterns in same group is chosen as the representation of all patterns in the group. The representative pattern of each group is used for evolution in the ART-MBDE algorithm.

4. Numerical results and discussions

In order to verify the performance of the proposed ART-MBDE algorithm, some studies selected from other researches were used in this study to demonstrate the feasibility of proposed algorithm. For parametric optimization, some benchmark functions with theoretical solution are used in comparison of results. Some cases of topology optimization of structure are applied for study the performance of ART-MBDE in nonparametric optimization problem.

4.1. Parametric optimization

The performance comparison of ART-MBDE algorithm with FADE and original MBDE algorithm is presented in this section. FADE, proposed by Liu et a1.[18], is based on real-valued space which using fuzzy logic controllers to adapt the search parameters for mutation and crossover operations. Four test functions in the literature are used for benchmark functions of numerical optimization are shown in Table 1. The population size and maximum number of generations used in ART-MBDE are the same as the settings used in FADE and MBDE. The parameter setting of ART-MBDE is listed in Table 2.

TABLE 1 TEST FUNCTIONS FOR COMPARISON

Test Function Range of variables fmiD

D (-5.12,5.12) 0.0 f1(x) = 2>�

j=l

1840


D-l (-2.048,2.047) 0.0 f 2(x) = �)100*(Xj_l _X�)2 +(1-xj)2)

}=1

D (-5.12,5.12) 0.0 f3(x) = L(jloor(xj + 0.5))2 j=1

D (-5.12,5.12) 0.0 f4(x) = �)(x� -10cos(2,,·xj)) +10·D j=l

TABLE 2 PAR AMETER SETTINGS OF ART-MBDE

Parameter Value Population size 50

Bit-String length 30 Fl mutation 0.5 F2 mutation 0.005

CR crossover 0.8 Maximum Generations 300 vigilance test threshold 0.75

A comparison of results for FADE, MBDE, and ART-MBDE is shown in Table 3. f is the index of the test function, while D and G represent the dimension and maximum number of generations the test function used respectively. The final result of ART-MBDE is better than MBDE across all test functions. Compared to FADE proposed in the literature, the solutions found by ART-MBDE are not superior to that of FADE only for test functions f4. But it is close to the global minimums of test function f4.

TABLE 3 COMPARISON RESULTS FOR TEST FUNCTIONS

f D G FADE MBDE f1 3 50 1.18xlO -5 1.82xl0 -12

f2 2 50 2.20xlO -3 4.41xlO -3 f3 5 30 0 6.93xlO -

2

f4 2 100 1.13xlO -7 1.87xlO -2

4.2. Minimum compliance design problem

ART-MBDE

1.78xl0 -12

4.63xlO -4

0

5.05xlO -3

. Topology optimization is useful for concept design

durmg development of an innovative product. When using the finite element method to analyze structure, each element without constraints can be eliminated or retained in the design domain. For two-dimensional structure, design domain of topology optimization is subdivided into elements for the finite element model. A binary string is used to store the shape information of structure. The code "I" in the string refers to the solid element of the structure while the code "0" in the string represents th� corresponding void element of the structure. The minimum compliance topology optimization can be expressed as:

Minimize C(x) = UTKU Subject to VNo = f (15) C(x) is the compliance of structure. U and K are the

global displacement and stiffness matrix. f is the prescribed volume fraction. V and Vo are the material volume and design domain volume, respectively. This example consists of minimizing a 2x1 cantilevered plate's compliance with materials limitations. All degrees of freedom are fixed for the nodes located at the top and bottom left, as shown in Fig. 3. The thickness is 1 and the force with a magnitude of one is applied at the center on the right side. The design domain is divided into 24x 12 quadrilateral elements for finite element analysis. The Young's modulus E is 1, and Poisson's ratio v is 0.3. The objective is to find the minimum compliance of the structure subjected to a maximum volume constraint of 50%. The parameter setting of ART-MBDE is listed in Table 4. Further, the topological result is also compared with the results obtained by MBDE and stress-enhanced MBDE.

TABLE 4 MINIMUN COMPLIANCE PAR AMETER SETTINGS

Parameter Value PODulation size 10

Bit-String length 288 Fl mutation 0.1 F2 mutation O.oI


Fig.3 Design domain for 2xl cantilever beam

Comparison of Fig. 4 shows that the optimum results of ART-MBDE and MBDE with and without using the stress-based binary mutation mechanism are very similar . The stress-based binary mutation obtains a best solution with fewer objective function calculations: 11,360 for the stress-based MBDE; 14,200 for ART-MBDE; 19,800 for MBDE. Although the result of ART-MBDE is worse than stress-enhanced MBDE result, but they are very close. Besides, ART-MBDE use binary pattern only without enhancement of stress information in stress-based MBDE. It is easier to adapt ART-MBDE for general optimization problems. The result of ART-MBDE is better than MBDE in both compliance and objective function calculation. Clearly ART clustering mechanism enhances the

1841


performance of ART-MBDE in solving topology structural optimization problems.

Fig. 4 Optimum results of (a) MBDE, compliance = 64.21 (b) ART-MBDE, compliance = 64.29 (c) MBDE with stress-based binary mutation, compliance = 64.21

4.3. Minimum weight design problem

The minimum weight topology optimization can be formulated as:

Minimize W(x)

Subject to d� dallow (J� (Jallow (6)

Where W(x) is the weight(volume) of structural topology, d and (J are the displacement and equivalent stress of topology design. dallow and (Jallow are corresponding allowable displacement and stress. The objective is to find a feasible topology design with minimum weight that satisfies the allowable displacement or stress requirement. The geometric dimensions of the cantilever structure are 1 in. width and 2 in. height. The design domain is divided into lOx20 quadrilateral elements for finite element analysis. It is subjected to a load F at center on the right side, and the value of force F is 1. It is assumed that the value of multiplication of Et is 1. E is the Young's modulus and t is the thickness of the plate. The maximum displacement is 20. The nodes with all degrees of freedom fixed are located at the top and bottom left, as shown in Fig. 5. The parameter setting of ART-MBDE is listed in Table 5.

TABLE 5 MINUM WEIGHT PARAMETER SETTINGS

Parameter Value Population size 10

Bit-String length 200 Fl mutation 0.1 F2 mutation 0.01


...

...

Fig. 5 Design domain for 2x1 cantilever beam

The optimum results of MBDE with/without stress-based binary mutation and ART-MBDE are shown in Fig. 6. Comparison of Fig. 6 shows that the optimum results of ART-MBDE and MBDE with stress-based binary mutation mechanism are very similar. The ART-MBDE obtains a best solution with fewer objective function calculations: 2,690 for the stress-based MBDE; 2,110 for ART-MBDE; 3,020 for MBDE. The result of ART-MBDE is better than MBDE and stress-based MBDE in both compliance and objective function calculation. Again ART clustering mechanism improves the performance of ART-MBDE III solving minimum weight optimization problems.

Fig. 6 Optimum results of (a) MBDE, weight=0.20 (b) ART-MBDE, weight=0.19 (c) MBDE with stress-based binary mutation, weight= 0.19

5. Conclusions

In this study, adaptive resonance theory is successfully integrated with modified binary differential evolution algorithm to solve different optimization problems. Comparing with the results of FADE and MBDE algorithms in function optimization problems, the optimized ART-MBDE results are close or superior to the FADE or MBDE results. For tackling structural topology optimization problems, the ART-MBDE can obtain better solutions with fewer objective function calculations weather in minimum compliance design problems or minimum weight design problems in comparison with MBDE. The ART-MBDE results are close or equal to the stress-based MBDE results. But only binary image pattern and clustering operation are applied in ART-MBDE for optimization search without help of other problem dependent information, like stress in stress-based MBDE. ART-MBDE has higher potential for solving general multiphysics optimization problems .

The clustering technique of adaptive resonance theory enhances the performance of MBDE algorithm and keeps good balance between exploration and exploitation during evolution process. The new developed ART-MBDE algorithm is suitable for not only dealing with real-valued parametric optimization problems, but also finding optimal solution of binary-coded topology optimization of structure. Judging from the results of different optimization problems

1842


illustrated in this study, including parametric optimization and structural topology optimization, ART-MBDE has high viability for optimization problems using real-valued or binary -coded representation.

References

[1] R. Storn and K. Price, "Differential Evolution-A simple and efficient adaptive scheme for global optimization over continuous space", Technical Report TR-95-012, 1995.

[2] R. Storn and K. Price, "Differential evolution-A simple and efficient heuristic for global optimization over continuous spaces", Journal of Global Optimization, vol. 11, 1997, pp. 341-359.

[3] C.C. Cheng, T. D. Tsai, D. W. Lin and C. C. Chiu, "Design and Analysis of Shock- Absorbing Structure for Flat Panel Display", IEEE transactions on advanced packaging, vol. 31,2008, pp.135-l42.

[4] B. Qian, L. Wang, D. X. Huang and X. Wang, "Scheduling multi-objective job shops using a memetic algorithm based on differential evolution", International Journal of Advanced Manufacturing Technology, vol. 35,2008, pp. 1014-1027.

[5] Y. Gao and J. Liu, "A modified differential evolution algorithm and its application in the training of B P neural network", IEEE/ ASME International Conference on Advanced Intelligent Mechatronics, Xi'an, China, July 2008, pp. 1373-1377.

[6] G. Pampani, A. P. Engelbrecht, and N. Franken, "Binary Differential Evolution", IEEE Congress on Evolutionary Computation, Sheraton Vancouver Wall Centre, Vancouver, BC, Canada, 2006, pp. 1873-1879

[7] H. Xingshi and H. Lin, "A novel binary differential evolution algorithm based on artificial immune system", IEEE Congress on Evolutionary Computation, Singapore, September 2007, pp. 2267-2272

[8] Kevin 1. Binkey & Masafumi Hagiwara, "Balancing Exploitation and Exploration in Particle Swarm Optimization: Velocity-Based Reinitialization", Transactions of the Japanese Society for Artificial Intelligence, Vol. 23, No. 1, pp. 27-35, 2008

[9] Marco Laumanns, Lothar Thiele, Kalyanmoy Deb, Eckart Zitzler, "Combining Convergence and Diversity in Evolutionary Multiobjective Optimization", Evolutionary Computation, Vol. 10, No. 3,pp. 263-282,2002

[10] Enrique Alba and Bernabe Dorronsoro, "The Exploration/Exploitation Tradeoff in Dynamic Cellular Genetic Algorithms", IEEE Transactions on Evolutionary Computation, Vol. 9, No .. 2, April 2005

[11] Shih-Hsi Liua, Marjan Mernikb and Barrett R. Bryanta, "To explore or to exploit: An entropy-driven approach for evolutionary algorithms", International Journal of Knowledge-based and Intelligent Engineering Systems, Vol. 13, pp. 185-206, 2009

[12] Mariem Gzara, Abdebasset Essabri, "Balanced Explore-Exploit Clustering-based Distributed Evolutionary Algorithm for Multi-objective Optimization", Studies in Informatics and Control, Vol. 20, No. 2, pp 98-106 , June 2011

[13] A. E. Irnrani, A. Bouroumi, H. Zine El Abidine, M. Limouri, and A. Essa"ld, "A fuzzy clustering-based niching approach to multimodal function optimization", Journal of Cognitive Systems Research, Vol. 1, pp. 119-133,2000

[14] Alessandro Passaro and Antonina Starita, "Particle Swarm Optimization for Multimodal Functions: A Clustering Approach", Journal of Artificial Evolution and Applications, Volume 2008

[15]Ling Qing, Wu Gang, Yang Zaiyue, Wang Qiuping, "Crowding clustering genetic algorithm for multimodal function optimization", Applied Soft Computing, Vol. 8, pp. 88-95,2008

[16] G. A. Carpenter and S. Grossberg, "A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine", Computer Vision

Graphics Image Processing, pp:37-54, 1987. [17] S. Grossberg, "Competitive learning: From interactive

activation to adaptive resonance," Cognitive Science, Vol. 11, pp. 23-63, 1987.

[18] J. Liu and 1. Lampinen, "A Fuzzy Adaptive Differential Evolution Algorithm", Soft Computing, Vol. 9, pp. 448-462, 2005

[19] C.Y. Wu and K.Y. Tseng, "Topology optimization of structures using modified binary differential evolution," Structural and Multidisplinary Optimization, Vol. 3, July 2010

[20] C.Y.Wu and K.Y. Tseng, "Stress-based binary differential evolution for topology optimization of structures", Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, Vol. 224, No. 2, pp. 443-457, 2010

1843

Documents

[IEEE 2012 International Conference on Machine Learning and Cybernetics (ICMLC) - Xian, Shaanxi, China (2012.07.15-2012.07.17)] 2012 International Conference on Machine Learning and