ARTICLE IN PRESS

0925-2312/$ - se

doi:10.1016/j.ne

�CorrespondE-mail addr

(Y. Zhou).

Neurocomputing 71 (2008) 584–591

www.elsevier.com/locate/neucom

A novel framework for automatic generation of fuzzy neural networks

Meng Joo Era, Yi Zhoub,�

aSchool of Electrical and Electronic Engineering, Nanyang Technological University, S1 Nanyang Ave, Singapore 639798, SingaporebSchool of Electrical and Electronic Engineering, Singapore Polytechnic, 500 Dover Road, Singapore 139651, Singapore

Received 30 January 2007; accepted 28 March 2007

Available online 29 September 2007

Abstract

In this paper, a novel framework for automatic generation of fuzzy neural networks (FNNs) termed hierarchically generated fuzzy

neural networks (HGFNN) is proposed for realizing machine intelligence. Human intelligence in organizing companies in a civic society

has been adopted in this framework. In the HGFNN framework, an FNN is regarded as a company and fuzzy rules are considered as

employees of the company. Analogous to the management of a company, three criteria, namely client satisfaction, performance

evaluation and cost minimization, have been proposed. Simulation studies on mobile robot control demonstrate that the proposed

method is superior to other existing approaches.

r 2007 Elsevier B.V. All rights reserved.

Keywords: Machine learning; Fuzzy neural networks; Structure identification; Q-learning

1. Introduction

Fuzzy neural networks (FNNs), combining the profoundlearning capability of neural networks (NNs) with themechanism of explicit and easily interpretable knowledgepresentation provided by fuzzy logic has attracted greatattention in the recent years. However, conventionalsubjective approaches for designing FNNs are either timeconsuming or not applicable if the system is too complex oruncertain. Therefore, a systematic way of designing FNNsis proposed in this paper.

Most of the existing machine learning techniques areconstructed either by statistic analysis [13] or withinspiration from bioscience [8,11,18,24,25]. The self-orga-nizing maps (SOM) [8] are inspired from studies of ahuman brain while Genetic Algorithms (GAs) [11] imitatenatural evolutionary approaches. Reinforcement learning(RL) [18] simulates animal learning by giving rewards orpenalties to right or wrong behaviors, respectively. Allthese methodologies are based on research on naturalscience. However, human being’s management skills are

e front matter r 2007 Elsevier B.V. All rights reserved.

ucom.2007.03.015

ing author. Tel.: +658707873.

esses: emjer@ntu.edu.sg (M.J. Er), zhouyi@sp.edu.sg

seldom mentioned for machine learning. Since humanbeings are the most advanced and intelligent mammal inthe world, the way how a human being manages companiesand societies can be adopted for enhancing machineintelligence. Human intelligence in organization of thecivic society, business and human resources can be adoptedas novel sources of inspiration for the development ofmachine learning. In this paper, human intelligence inmanagement is implemented for generating FNNs and ahierarchical framework termed hierarchically generatedfuzzy neural networks (HGFNN) is proposed for all typesof learning approaches. In the HGFNN, generation ofFNNs is viewed analogously to the management ofcompanies in the civil society. In the HGFNN, determina-tion of structure and premises of FNNs is regardedanalogously to managing the structure and humanresources in a company. At the same time, consequentparameters of FNNs are estimated by individual learningprocesses of local rules. This is analogous to an employeewho desires to improve his/her performance throughpractice and learning.There are three main classes of machine learning

techniques, which are capable of generating and trainingthe structure and parameters of FNNs, namely supervisedlearning (SL), RL and unsupervised learning (UL). SL is a

ARTICLE IN PRESSM.J. Er, Y. Zhou / Neurocomputing 71 (2008) 584–591 585

learning process which requires a set of input–outputtraining data and attempts to learn the input–outputrelationship f ðxÞ by using the training data set. A numberof SL approaches for generating FNNs are proposed[5,21,22]. However, the training data are not alwaysavailable especially when a human being has little knowl-edge about the system or the system is uncertain. In thosesituations, UL and RL are preferred over SL as ULand RL are learning processes that do not need anysupervisors to tell the learner what actions to take.A number of researchers have applied RL [18] to trainthe consequent parts of an FNN [1,6,7] and severalworks combining the SL and RL, which use SL for offlinelearning and RL for online learning, were proposed in[2,23]. In this paper, we focus on RL in the proposedHGFNN framework and the Q-learning of [20] isadopted as a specific learning technique since it is the mostpopular RL approach. Simulation studies with existingQ-learning-based methodologies on mobile robot controlare carried out to illustrate the efficiency of the proposedapproach.

The organization of this paper is as follows: Thearchitecture of the HGFNN system is introduced inSection 2, while the philosophy of the HGFNN frameworkis presented in Section 3. Methodologies for structuregeneration via HGFNN are proposed in Section 4.Simulation studies and comparisons are included in Section5 while concluding remarks are given in Section 6.

2. Architecture of the HGFNN system

If the output linguistic variables of a multi-input multi-output (MIMO) fuzzy system are independent, MIMOFNNs can be represented as a collection of multi-inputsingle-output (MISO) FNNs by decomposing the aboverule into m sub-rules with Gj ; j ¼ 1; 2; . . . ;m as the singleconsequent of the jth sub-rule [19]. The architecture of the

x1

xn

f1

fl

w1

wl

y

Layer 1input

Layer 2rule

Layer 3normalization

Layer 4output

x2

�l

�1

Fig. 1. Structure of the HGFNN system.

HGFNN system is based on extended EBF NNs, which arefunctionally equivalent to the TSK fuzzy system [5]. TheNNs structure of the HGFNN for an MISO system isdepicted in Fig. 1.In the four-layer architecture, layer one is an input layer

and layer two is a rule layer. The two layers formulate thepremises and structure of FNNs. While layer three is thenormalization layer and layer four is an output layer.A fuzzy rule in this architecture is expressed as follows:

If x1 is ðc1i;s1iÞ and x2 is ðc2i;s2iÞ and . . . xn is ðcni; sniÞ

Then y is oi;

where ðc;sÞ are the center position and width of fuzzyneurons and o is the weight of fuzzy neurons in the system.Here, ðc;sÞ are regarded as premise parameters while o isconsidered as consequent parameters. The membershipfunctions (MFs) are usually adopted with Gaussianfunctions and multiplication is selected as the T-normoperator. If the center-of-gravity (COG) method isperformed for defuzzification, the output variable, as aweighted summation of the incoming signals, is given by

y ¼XL

j¼1

fjoj, (1)

where fj is the normalized firing strength of the jth rule.Other MFs, such as triangular MFs, and other T-norm

operators are also applicable in the HGFNN.In this paper, structure identification and estimation of

parameters are achieved via a systematic evaluation whichis analogous to managing a company in a civil society.

3. Philosophy of the HGFNN framework

From the viewpoint of structure and management, thereare many common points between the management of acompany in a civic society and the generation of FNNs inthe engineering field. Firstly, both companies and FNNshave structures to be identified. The manpower strengthhas to be defined in a company and the number of fuzzyrules has to be decided in the FNNs analogously. More-over, both premise and consequent parameters of FNNshave to be estimated which is analogous to assigningauthority and function to each employee in a company.In short, both the companies and FNNs are configurablein their structures. Secondly, both companies andFNNs are target-oriented. From an economic viewpoint[4], the target of a company is to acquire maximumrevenue with the lowest cost, which can be regarded asmaximizing the profits of the company. At the sametime, the target of training FNNs is also to optimizecertain objective functions. Therefore, both the companiesand FNNs are performance-oriented. Lastly, the organiza-tion of a company has to be optimized and adjustedaccording to the market and performance. Analogously,FNNs are to be trained for optimizing its structureand improving the performance. In summary, both the

ARTICLE IN PRESS

Table 1

Analogous comparison between the HGFNN strategy and management of

a company

HGFNN strategy Management

FNN Company

Fuzzy neurons Employees

Inputs Clients

Objective functions Profits

Center (MF) Position

Width (MF) Responsibility

Firing strength Participation

Consequent weight Employee’s action/service

M.J. Er, Y. Zhou / Neurocomputing 71 (2008) 584–591586

companies and FNNs are reconfigurable, performance-oriented and require optimization. Therefore, humanintelligence in managing a company can also be adoptedfor generating FNNs.

In the HGFNN framework, the entire FNN is treated asa large organization or a company and each fuzzy rule isregarded as an agent or an employee. The inputs of anFNN are regarded as the clients of a company. The widthof a fuzzy neuron is regarded analogously to the weight orthe authority of an employee in a company. The purpose oftraining an FNN is to achieve a maximum objectivefunction which is similar to acquiring maximum profit in acompany. In a civic society, people always hold certainexpectation of a company and several requirements to theemployees. Analogically, the performance of FNNs isevaluated as a company is evaluated by people. Fuzzyrules and neurons are to be created, adjusted and deletedas employees are recruited, reallocated and fired in acompany.

An analogous comparison between the HGFNN and themanagement of a company is listed in Table 1.

4. Structure generation via the HGFNN framework

In managing a company, a certain number of issues needto be addressed by a manger for determining the size andallocation of manpower. As stated in [17], the core purposeof an organization may be stated as the creation of goodsor services for customers (clients). There are three criteriafor managing a company, namely client satisfaction,performance evaluation and cost minimization. The clientsatisfaction criterion requests that all clients must be well-served and the performance evaluation criterion ensuresthat a company with unsatisfactory performance will berestructured and employees are to be reinforced orpunished according to their individual performances. Atthe same time, the production cost should be reducedwithout affecting the performance. Analogously, a well-generated FNN should partition the entire input space welland obtain satisfactory objective function with efficientcomputation. Therefore, the three criteria are proposedanalogously for generating FNNs.

4.1. Client satisfaction criterion

The size of a company is determined by the size of themarket and the allocation of the clients. If there are noclients or customers existing in the area, no business will bedone. On the other hand, if there are some clientsunattended, more agents should be recruited to serve theclient. An accommodation boundary criterion can beadopted according to the client satisfaction criterion.A fuzzy neuron is a class of representations over a region

defined in the input space. If a new sample falls within theaccommodation boundary of one of the currently existingfuzzy neurons, the HGFNN will not generate a new neuronbut accommodate the new sample by updating theparameters of existing fuzzy neurons. However, if the inputdoes not fall within any accommodation boundary of theexisting neurons, a new rule will be created. The accom-modation boundary can be checked via the �-completenesscriterion as in [22]. Besides the �-completeness criterion,other methodologies, such as the SOM [8], growing neuralgas (GNG) networks [3], can also be adopted to adjust theFNN according to the input space (clients).

4.2. Performance evaluation criterion

Once there are sufficient manpower to service all theclients, the quality of the service should be evaluated.Analogously, performances of the FNNs are examinedafter the input space has been well-covered. In theHGFNN framework, performances of both the entiresystem and individual fuzzy rules are to be evaluated.

4.2.1. Evaluation of system performance

Once all clients are serviced by the agents, the systemperformances should be evaluated. There are various waysof evaluating the system performance depending on thetask requirements and the training technologies. If SL isapplied, the system is evaluated by the classificationaccuracy. In the RL approaches, the system performancesare examined by the reward values which are definedaccording to the training objectives. If the system error inSL is regarded as a penalty function in RL, SL can beregarded as a special case of RL by

reward ¼ Ke � keik, (2)

where Ke is a positive value and set according to the desiredaccuracy.In this paper, we focus on RL approaches and criteria

based on the reward functions are proposed. Averagerewards are adopted for evaluating the system performanceif the training environment is static. However, discountedrewards are to be considered for dynamic environment, i.e.

ravg ¼

Pt¼Tt¼0 rðtÞ

TStatic environment;

Pt¼Tt¼0 g

trðtÞ

TDynamic enviroment;

8>>><>>>:

(3)

ARTICLE IN PRESSM.J. Er, Y. Zhou / Neurocomputing 71 (2008) 584–591 587

where T is the training time of each episode, rðtÞ is thereward obtained at time t and g is the discounted factorwhich is less than 1.

4.2.2. Evaluation of individual performance

In every company, there are appraisal evaluations forevery employee. There are a number of means of evaluatingcontributions of fuzzy neurons such as the competitivelearning [9] and the error reduction ratio (ERR) method[22] for SL approaches. For RL, the local contributions offuzzy rules can be measured simultaneously by the localrewards which are given, as in [15], as follows:

rjlocalðtÞ ¼ fjrðtÞ; j ¼ 1; 2; . . . ;L, (4)

where fj is the normalized firing strength of the jth rule andr is the system reward.

4.2.3. Structure generation based on performance criteria

If the global average reward value is less than athreshold, kg, it means that the overall performance isunsatisfactory. To resolve this problem, the weights ofsome good rules should be increased which means that thesystem will be modified by promoting the rule with the bestperformance, e.g. best local reward to a more importantposition. As a result, the overall performance is improvedas the rules with good contributions (analogously, employ-ees with good performance) participate more in the systemwhile those rules with poor contributions (analogously,employees with poor performance) participate less corre-spondingly. In this case, the width of the jth fuzzy rule’sMF (the one with the best local reward) will be increased asfollows:

sij �ksij ; i ¼ 1; 2; . . . ;N if ksijosmax

�smax if ksij4smax, ð5Þ

where k is slightly larger than 1 and smax is the maximinwidth allowed for Gaussian MFs.

Remarks. Restructuring is only adopted when the systemunder-performs; enlarging the MF of the rule will increasethe firing strength of the rule. As a result, global action isimproved by becoming closer to the best local action.

On the other hand, if the local reward is less than a heavythreshold value, klh, the rule will be deleted. If the localreward of the rule is larger than klh but smaller than a lightthreshold value, kll, a punishment will be given to the ruleby decreasing its width of the MF as follows:

sik �tsik; i ¼ 1; 2; . . . ;N, (6)

where t is a positive value less than 1.

Remarks. This is a performance-oriented approach, inwhich fuzzy rules with the best local reward are to beenlarged and those with unsatisfactory rewards are to bereduced or even removed from the system. It is analogousto the promotion and demotion strategy in human resourcemanagement.

Variolous performance-oriented strategies, such as theGA, can also be adopted in the HGFNN framework.

4.3. Cost minimization criterion

In order to minimize the cost, redundant employees areeliminated from the company. Similarly, redundant fuzzyneurons are to be removed for reducing the computationcost, if they are not active or sensitive. The sensitivitycalculation method [14], weight decay method [14] andminimal output method [10] are SL pruning techniques forreducing the computational cost. In this paper, an averagefiring strength is proposed as a measurement for participa-tion. If a fuzzy rule has very low firing strength during theentire episode or a long period of time recently, the rule isto be eliminated.

4.4. Gradualism of learning of a company

It is natural to set the profit margin lower at the earlierstage and increase it at a later stage. Analogous to this,gradualism learning should be adopted for generatingFNNs. In the HGFNN method, the demanding require-ment is set low at the beginning and increased higher laterwhen the training becomes stable, i.e.

k ¼ kminþ ðkmax

� kminÞ

run

runþ Kr

, (7)

where kmin and kmax are the minimal and maximal valuesfor thresholds, respectively, run is the number of trainingepisodes or period, and Kr is a controlling constant whichcan be set according to the total number of runningepisodes.

5. Simulation studies

An HGFNN strategy is proposed in this paper and it canbe applied with various training technologies. In this paper,Q-learning is adopted for estimating the consequentparameters. The Q-learning was applied in generating theconsequent parameters of FNNs in the fuzzy Q-learning(FQL) [6] and the FQL is also adopted in the dynamicfuzzy Q-learning (DFQL) [1] and the online clustering andQ-value-based GA learning schemes for fuzzy systemdesign (CQGAF) [7] for self-generated FNNs. Thoseapproaches have been applied to control a Khepera mobilerobot of [16] for a wall-following task.The Khepera mobile robot is the same as that in [1]

which is shown in Fig. 2 and the distance between the robotand the obstacle can be derived from the sensor values.Similar to the experiment in [1], the task is to design acontroller for wall-following, while the robot is onlymoving in clockwise direction at a fixed speed. Four inputvariables, which are the values of sensors, Si ði ¼ 0; 1; 2; 3Þ,are considered and all values are normalized. The output ofthe controller is the steering angle of the robot.

ARTICLE IN PRESS

Fig. 2. Position and orientation of sensors on the Khepera II.Fig. 3. Training environment.

M.J. Er, Y. Zhou / Neurocomputing 71 (2008) 584–591588

The aim of the wall-following task is to control themobile robot to follow a wall while keeping a distance fromthe wall in the range of ½d_; dþ�. The same reward functionas what is used in [1] is adopted:

r ¼

0:1 if ðd_ododþÞ and ðU 2 ½�8�;þ8��Þ;

�3:0 if ðdpd_Þ or ðdþpdÞ;

0:0 otherwise:

8><>:

(8)

Here, U is the steering angle, d_ ¼ 0:15 and dþ ¼ 0:85,which are normalized values, are the same as the valuesset in [1].

The simulation version of the Khepera robot reportedin [12] is used for comparison studies. A complicatedenvironment which is shown in Fig. 3 is adopted.

The set of discrete actions is given by A ¼ ½�30;�25;�20;�15;�10;�5; 0; 5; 10; 15; 20; 25; 30� and otherparameters of the Q-learning algorithm are set as normalvalues: discounted factor, g ¼ 0:95; trace-decay factor,l ¼ 0:7; exploration rate, Sp ¼ 0:001; temporal difference(TD) learning rate, a ¼ 0:05. For the DFQL, the samesetting as in [1] has been adopted, i.e. �-completeness,� ¼ 0:5; similarity of MF, kmf ¼ 0:3; TD error factorK ¼ 50; TD error criterion, ke ¼ 1. The CQGAF proposedin [7] utilizes GA to train the consequent parameters of theFNN. For a fair comparison, fixed action set is applied forthe CQGAF as other approaches do not utilize GA forexhaustive search. As the aligned clustering algorithm issimilar to the �-completeness criterion in the DFQL butdescribed in another way, all parameters for the alignedclustering algorithm are set the same as those listed abovefor the DFQL (no TD error criterion in the CQGAF).For the proposed HGFNN method, the training aimis to limit the number of failures to 50. Therefore, thethreshold values of the reward criterion should be setaround ð�3� 50=1000Þ ¼ �0:15. If it requires eachrule to be active at least with an average firing strengthfor 10 times in the 1000 training steps among about 50rules, the firing strength threshold value is set as 10=ð1000�50Þ ¼ 0:0002. If gradualism learning is adopted, thethreshold values can be set around those values, e.g. thesystem reward thresholds value are kmax

g ¼ �0:05 andkming ¼ �0:45; heavy local threshold values for elimination

are kmaxlh ¼ �0:30 and kmin

lh ¼ �0:10; light local reward

threshold values for demotion are kmaxll ¼ �0:20 and kmin

ll ¼

0 and the control constant for gradualism learning isK r ¼ 20, which is half of the number of total trainingepisodes. The promotion constant is k ¼ 1:05 and thedemotion constant is t ¼ 0:95. These values give goodperformances of the algorithms in an initial phase.However, it should be pointed out that we have notsearched the parameter space exhaustively.The performances of these different approaches are

evaluated at every episode of achieving 1000 successfulsteps. Three criteria, namely number of failures, rewards

and number of rules, are considered for measuring theperformance. The first two criteria are to measure how wellthe task has been performed and the last one is to measurehow much computational cost has been utilized. Theperformances of these three approaches have been mea-sured and the mean values for 40 episodes over 10 runshave been presented in Fig. 4. The mean values of thefailure, reward and number of fuzzy rules after 20 episodesare listed in Table 2. It is expected that these learningsystems should achieve satisfactory performances after 20training episodes and the mean values of the last 20episodes are chosen as the measurement for performances.Comparative studies show that the HGFNN approach is

superior to the DFQL and the CQGAF as the number ofrules is much fewer while similar or even better perfor-mances have been achieved.In order to demonstrate dynamic adaptivity to different

environments, similar to [1], a new environment shown inFig 5 is selected for training. Instead of training fromrandom initial values, the already trained FNNs from oldtraining is embedded. This also demonstrates how thecontroller responds when the environment is dynamicallychanged.The performances of the robot in the new environment

by training directly and re-training with the FNNsgenerated in the previous environment are shown in Fig. 6.During re-training, the robot is re-trained in the new

environment with the obtained navigation strategies fromthe previous training. From Fig. 6, it is clear that goodadaptive capability of the HGFNN approach has beenachieved. Some redundant rules are removed while somenew rules are generated. As a result, the robot is able toachieve satisfactory performance with the pre-obtained

ARTICLE IN PRESS

0 5 10 15 20 25 30 35 4020

25

30

35

40

45

50

55

60

Episodes

Num

ber

of ru

les

HGFNN

CQGAF(without GA)

DFQL

0 5 10 15 20 25 30 35 4040

60

80

100

120

140

160

180

200

220

240

Episodes

Num

ber

of fa

ilure

s

HGFNN

CQGAF(without GA)

DFQL

0 5 10 15 20 25 30 35 40−700

−600

−500

−400

−300

−200

−100

0

Episodes

Rew

ard

s

HGFNN

CQGAF(without GA)

DFQL

Fig. 4. Comparison of performances of DFQL, CQGAF and HGFNN in the wall-following task. (a) Number of failures vs number of episodes.

(b) Reward values vs number of episodes. (c) Number of rules vs number of episodes.

Table 2

Mean value of failures, reward and fuzzy rules after 20 episodes

DFQL CQGAF (without GA) HGFNN

Failure 77.54 61.85 52.32

Reward �148.01 �100.20 �74.49

Rules 57 44 27

Fig. 5. New training environment.

M.J. Er, Y. Zhou / Neurocomputing 71 (2008) 584–591 589

FNNs and is able to adapt its previous knowledge to thenew environment quickly.

6. Conclusions

In this paper, a novel HGFNN framework for generat-ing FNNs as a new approach in machine learning toimitate human management skills is proposed. Differentfrom the statistical analysis and biological and evolution-ary inspired methods, FNNs are generated analogously tomanaging a company in the civil society. The HGFNNregards determination of structure and premises of FNNsas managing the structure and human resources in acompany. Three criteria, named client satisfaction, perfor-mance evaluation and cost minimization, are proposedanalogously to the organizing skills in management. At themean time, consequent parameters of FNNs are estimatedby the individual learning process. Human intelligence insocial science can be adopted to train the structure ofFNNs which is a natural scientific problem. By this means,techniques and strategies towards managing companiesand generating FNNs are integrated. Simulation results

ARTICLE IN PRESS

0 5 10 15 20 25 30 35 4040

60

80

100

120

140

160

180

200

Episodes

Num

ber

of fa

ilure

s

Training directly

Re–training

0 5 10 15 20 25 30 35 40−600

−500

−400

−300

−200

−100

0

Episodes

Rew

ard

s

Training directly

Re–training

0 5 10 15 20 25 30 35 4024

25

26

27

28

29

30

31

32

33

34

Episodes

Num

ber

of ru

les

Training directly

Re–training

Fig. 6. Performance comparison with training directly and retraining in a new environment. (a) Number of failures vs number of episodes. (b) Reward

values vs number of episodes. (c) Number of rules vs number of episodes.

M.J. Er, Y. Zhou / Neurocomputing 71 (2008) 584–591590

demonstrate the superiority of the proposed method. Aglobal framework is proposed in this paper and technol-ogies applicable in the framework are still open issues.

References

[1] M.J. Er, C. Deng, Online tuning of fuzzy inference systems using

dynamic fuzzy Q-learning, IEEE Trans Syst. Man Cybern. Part B 34

(3) (2004) 1478–1489.

[2] M.J. Er, C. Deng, Obstacle avoidance of a mobile robot using hybrid

learning approach, IEEE Trans. Ind. Electron. 52 (3) (2005) 898–905.

[3] B. Fritzke, A growing neural gas network learns topologies, in: G.

Tesauro, D.S. Touretzky, T.K. Leen (Eds.), Advances in Neural

Information Processing Systems, vol. 7, MIT Press, Cambridge, MA,

1995, pp. 625–632.

[4] D. Gary, Organization Theory: Integrating Structure and Behavior,

Prentice-Hall, Englewood Cliffs, NJ, 1986.

[5] J.S.R. Jang, Anfis: adaptive-network-based fuzzy inference system,

IEEE Trans. Syst. Man Cybern. 23 (3) (1993) 665–684.

[6] L. Jouffe, Fuzzy inference system learning by reinforcement methods,

IEEE Trans. Syst. Man Cybern. Part C 28 (3) (1998) 338–355.

[7] C.F. Juang, Combination of online clustering and Q-value based GA

for reinforcement fuzzy systems, IEEE Trans. Fuzzy Systems 13 (3)

(2005) 289–302.

[8] T. Kohonen, Self-organized formation of topologically correct

feature maps, Biol. Cybern. 43 (1982) 59–69.

[9] C.T. Lin, C.S.G. Lee, Neural-network-based fuzzy logic control

and decision system, IEEE Trans. Comput. 40 (12) (1991)

1320–1336.

[10] Y. Lu, N. Sundararajan, P. Saratchandran, A sequential learning

scheme for function approximation using minimal radial basis

function neural networks, Neural Comput. 9 (2) (1997) 461–478.

[11] Z. Michalewicz, Genetic algorithmsþ data structures ¼ evolution

programs, third ed., Springer, London, UK, 1996.

[12] T. Nilsson, [Online]. Available: hhttp://www.kiks.f2s.comi.

[13] E. Oja, A simplified neuron model as a principal components

analyzer, J. Math. Biol. 15 (1982) 267–273.

[14] R. Reed, Pruning algorithms—a survey, IEEE Trans. Neural

Networks 4 (5) (1993) 740–747.

[15] P. Ritthiprava, T. Maneewarn, D. Laowattana, J. Wyatt, A modified

approach to fuzzy-Q learning for mobile robots, in: Proceedings of

the IEEE International Conference on Systems, Man and Cyber-

netics, vol. 3, 2004.

[16] S.A. K-Team, Khepera 2 User Manual, Switzerland, 2002.

[17] J.R. Schermerhorn, J.G. Hunt, R.N. Osborn, Organizational

Behavior, Wiley, New York, USA, 2005.

[18] R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction,

The MIT Press, Cambridge, MA, 1998.

[19] L.X. Wang, A Course in Fuzzy Systems and Control, Prentice-Hall,

NJ, USA, 1997.

ARTICLE IN PRESSM.J. Er, Y. Zhou / Neurocomputing 71 (2008) 584–591 591

[20] C.J.C.H. Watkins, P. Dayan, Technical note Q-learning, Mach.

Learn. 8 (3–4) (1992) 279–292.

[21] S. Wu, M.J. Er, Dynamic fuzzy neural networks: a novel approach to

function approximation, IEEE Trans. Syst. Man Cybern. Part B 30

(2) (2000) 358–364.

[22] S. Wu, M.J. Er, Y. Gao, A fast approach for automatic generation of

fuzzy rules by generalized dynamic fuzzy neural networks, IEEE

Trans. Fuzzy Systems 5 (4) (2001) 578–594.

[23] C. Ye, H.C. Yung, D. Wang, A fuzzy controller with supervised

learning assisted reinforcement learning algorithm for obstacle

avoidance, IEEE Trans. Syst. Man Cybern. Part B 33 (1) (2003) 17–27.

[24] L.A. Zadeh, Fuzzy sets, Inf. Control 8 (1965) 338–358.

[25] J.M. Zurada, Introduction to Artificial Neural Systems, West

Publishing Co., St. Paul, MN, USA, 1992.

Meng Joo Er received his B. Eng. and M. Eng.

degrees in Electrical Engineering from the

National University of Singapore in 1985 and

1988, respectively, and a Ph.D. in Systems

Engineering from the Australian National Uni-

versity in 1992. From 1987 to 1989, he worked as

a Research and Development Engineer in Char-

tered Electronics Industries Pte Ltd and a Soft-

ware Engineer in Telerate Research and

Development Pte Ltd, respectively. He served as

Director of the Intelligent Systems Centre, a University Research Centre

co-funded by Nanyang Technological University (NTU) and Singapore

Engineering Technologies from 2003 to 2006. Currently, he is an Associate

Professor in the School of Electrical and Electronic Engineering (EEE),

NTU. His research interests include control theory and applications, fuzzy

logic and neural networks, computational intelligence, cognitive systems,

robotics and automation, sensor networks and biomedical engineering. He

has authored two books, seven book chapters and more than 300 refereed

journal and conference papers in his research areas of interest. Dr. Er was

the winner the Institution of Engineers, Singapore (IES) Prestigious

Publication (Application) Award in 1996 and IES Prestigious Publication

(Theory) Award in 2001. He served as an Editor of IES Journal on

Electronics and Computer Engineering from 1995 to 2004. Currently, he

serves as an Associate Editor of four refereed international journals,

namely IEEE Transactions on Fuzzy Systems, International Journal of

Fuzzy Systems, Neurocomputing and International Journal of Humanoid

Robots.

Yi Zhou received his B.Eng. degree (first-class

honours) in Electrical and Electronic Engineering

from the Nanyang Technological University,

Singapore in 2004. He is currently appointed as

a lecturer at Singapore Polytechnic and working

towards his Ph.D. at NTU. His research interests

include machine learning, fuzzy logic, neural

networks, intelligent control, reinforcement

learning and robotic applications. Yi Zhou

received a Bronze EEE Color Award for excellent

achievements in both academe and activities in NTU and obtained

certificate of merit for publication award (student category) by the

Institution of Engineers, Singapore in 2004.