Review ArticleA Brief Review of Neural Networks Based Learning andControl and Their Applications for Robots

Yiming Jiang1 Chenguang Yang1 Jing Na2 Guang Li3 Yanan Li4 and Junpei Zhong5

1Key Laboratory of Autonomous Systems and Networked Control College of Automation Science and EngineeringSouth China University of Technology Guangzhou China2Faculty of Mechanical and Electrical Engineering Kunming University of Science and Technology Kunming China3School of Engineering and Material Sciences Queen Mary University of London London UK4School of Engineering and Informatics University of Sussex Brighton UK5National Institute of Advanced Industrial Science and Technology Tokyo Japan

Correspondence should be addressed to Chenguang Yang cyangieeeorg

Received 15 June 2017 Accepted 1 October 2017 Published 31 October 2017

Academic Editor Thierry Floquet

Copyright copy 2017 Yiming Jiang et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

As an imitation of the biological nervous systems neural networks (NNs) which have been characterized as powerful learning toolsare employed in a wide range of applications such as control of complex nonlinear systems optimization system identificationand patterns recognition This article aims to bring a brief review of the state-of-the-art NNs for the complex nonlinear systems bysummarizing recent progress of NNs in both theory and practical applications Specifically this survey also reviews a number ofNN based robot control algorithms including NN based manipulator control NN based human-robot interaction and NN basedcognitive control

1 Introduction

In recent years the research of neural network (NN) hasattracted great attention It is well known that mammalsrsquobrain which consists of billions of interconnected neuronshas the ability to deal with complex and computationallydemanding tasks such as face recognition body motionplanning and muscles activities control Figure 1 shows acellular structure of a mammalian neuron Inspired by theneuron structure artificial NN (ANN) was developed toemulate the learning ability of the biological neurons system[1ndash3] The concept of artificial NNs was initially investigatedbyMcCulloch and Pitts in the 1940s [3] where the network isestablished with a parallel structureThe basically mathemat-ical model of NN consists of three layers that is input layerhidden layer and output layer which are of simple parallelcomputational structure but with appealing learning abilityand computational power to predict nonlinear dynamicpatterns

In past decades the NN technique has been studiedextensively in areas such as control engineering aerospacemedicine automotive psychology economics energy sci-ence and many other fields [4ndash7] It has been reported thatNN can approximate any unknown continuous nonlinearfunction by overlapping the outputs of each neuron More-over the approximation errors could be made arbitrarilysmall by choosing sufficient neurons This enables us to dealwith control problems for complex nonlinear systems [8ndash13]In addition to systemmodeling and control NNhas also beensuccessfully applied in various fields such as learning [14ndash17] pattern recognition [18] and signal processing [19] AndNN has been extensively used for functions approximationsuch as to compensate for the effect of unknown dynamics innonlinear systems [20ndash31] The NN control has been provedto be effective for controlling uncertain nonlinear systemsand demonstrated superiority in many aspects

Recently the researchers have focused on the study ofrobotics for its increasing importance in both industrial

HindawiComplexityVolume 2017 Article ID 1895897 14 pageshttpsdoiorg10115520171895897

2 Complexity

Cell body

Axon terminals

Myelin sheath

NodesAxonDendrites

Nucleus

Figure 1 An example of mammalian neuron (modified from [32])

applications and daily life [33ndash38] Many advanced robotssuch as YuMi made by ABB Baxter made by Rethinkand Rolinsrsquo Justin developed by German Aerospace Agency(DLR) have also been widely allocated The robots manipu-lator system is characterized with high-nonlinearity strongcoupling and time-varying dynamics thus controlling arobot with not only positioning accuracy but also enoughflexibility to complete a complex task became an interestingyet challenge work To achieve a high performance controldynamics of the robot should be known in advance Howeverin practice the robot dynamic model is often rarely knowndue to the complex robot mechanism let alone variousuncertainties such as parametric uncertainties or modelingerrors existing in the robot dynamics Therefore advancecontrol algorithm is imperative for next-generation robotsThanks to the universal approximation and learning abilitythe NN has been widely applied in robot control with variousapplications The combination of NN and robot controllercan provide possible solutions for complex manipulationtasks for example robot control with unknown dynamicsand robot control with unstructured environment In thispaper we present a brief review of robot control by means ofneural network The rest of the paper is organized as follows

After the introduction in Section 2 we present prelim-inaries of several popular neural network structures suchas RBFNN and CMAC NN Section 3 introduces a numberof theoretical developments of NN in the fields of adaptivecontrol optimization and evolutionary computing In addi-tion Section 4 revisits the robot neural network control withthe applications in manipulation human-robot interactionand robot cognitive control Section 5 gives a brief discussionabout the neural network control and its future research

2 Preliminaries of Neural Networks

In this section we will introduce several types of NNstructure which are popularly employed in the controlengineering

21 Radial Basis Function Neural Network (RBFNN) [14 15]A basic architecture of RBFNN network is shown in Figure 2which consists of three layers namely input layer hiddenlayer and output layer In the input layer the NN inputsare applied In hidden layer the data is transformed frominput space to hidden space which is always with a higher

dimension The RBFNN can be used to approximate anycontinuous vector function for example 119865(119885)

119865 (119885) = 119882119879119878 (119885) (1)

where 119865(119885) is the estimation of 119865(119885) and 119885 is NN inputsvector = [1 2 119899] isin 119877119899times119897 is the estimation ofNN optimal weight 119878(119885) = [1199041(119885) 1199042(119885) 119904119897(119885)]119879 is theregressor and 119897 denotes the number of NN nodes Generallythe regressor could be chosen as a Gaussian radical basisfunction as follows

119904119894 (1003817100381710038171003817119885 minus 1199061198941003817100381710038171003817) = exp[minus (119885 minus 119906119894)119879 (119885 minus 119906119894)1205902119894 ] (2)

where 119906119894 (119894 = 1 119897) are distinct points in state space and120590119894 is the width of Gaussian membership function It has beenwell recognized that using the powerful approximate abilityof the RBFNNwe can approximate any continuous nonlinearfunction over a compact set as

119865 (119885) = 119882lowast119879119878 (119885) + 120576 (3)

where 119882lowast is the optimal weight vector and 120576 is the approxi-mate error

22 Cerebellar Model Articulation Controller (CMAC) NN[39] There has been a predominant tendency to study thelearning and control techniques of robots by exploring theprinciples of biological systems This is because the biolog-ical creatures mechanisms and underlying principles arelikely to bring novel ideas to improve control performanceof the robot in a complex environment In 1972 Albusproposed a learning mechanism that imitates the structureand function of the cerebellum called cerebellar modelarticulation controller (CMAC) which is designed based ona cerebellum neurophysiological model [40] In comparisonto the backpropagation neural network the CMAC NN wasadopted widely in modeling and control of robots system forits rapid learning speed simple structure insensitivity of datasequence and easy implementation [39 41]

Figure 3 shows the basic structure of the CMAC neuralnetwork The CMAC could be used to approximate theunknown continuous function 119866(119885) = [1198911(119885) 1198912(119885) 119891119899(119885)] where 119885 isin 119877119898 denotes the 119898 dimensional inputs

Complexity 3

Hidden layer

Output layerInput layer

Σ

Σ

z1

z2

zm

w11

w21

w31w2n w1n

w3n

wl1wln

f1(z)

fn(z)

Figure 2 Structure of the RBFNN

z1

zm

Input space

Association memory space

Weight memory space

Output space

Receptive field space

Σ

Σ

f1(z)

fn(z)

Figure 3 Structure of a CMAC neural network

space As shown in Figure 3 two components are involvedin the CMAC neural network to determine the value of theapproximated nonlinear function 119866(119885)

119877 119885 997888rarr 119862119875 119862 997888rarr 119865 (4)

where

119885 ism-dimensional input space

F is n-dimensional output space

C is119873119888-dimensional association space

and 119877(sdot) denotes the mapping from the input vector tothe association space that is 120572 = 119877(119885) The outputsare computed through 119875(120572) by using a projection of theassociation vector 120572 onto a weights vector such that

119891 = 119875 (120572) = 119882119879120572 (5)

It should be noted that 119877(119885) can be represented by amultidimensional receptive filed function such that eachpoint in input 119885 is assigned with an activation value The

receptive-field basis functions of the association vector couldbe chosen as Gaussian functions as follows

ℎ119894119896 (1003817100381710038171003817119911119894 minus 1199061198941198961003817100381710038171003817) = exp[minus (119911119894 minus 119906119894119896)21205992119894119896 ] 119896 = 1 2 119897

(6)

where l is number of blocks of the associate space ℎ119894119896denotes the kth block associated with the input 119911119894 119906119894119896denotes the receptive fieldrsquos center and 120599119894119896 is the varianceof Gaussian function Then the multidimensional receptive-field function can be described as

119878 (119885) = [1199041 1199042 119904119899]119879 (7)

where 119904119896(119885 119906119896 120599119896) = prod119898119894=1ℎ119894119896(119911119894) 119906119896 = [1199061119896 1199062119896 119906119898119896]119879and 120599119896 = [1205991119896 1205992119896 120599119898119896]119879 The following property showsthe approximation ability provided by the CMAC neuralnetwork

Lemma 1 For a continuous nonlinear function 119865(119885) thereexists an ideal weight value 119882lowast such that 119865(119885) could beuniformly approximated by a CMAC with the multiplicationof the optimal weights119882lowast and the associate vector 119878(119885) as

119865 (119885) = 119882119879119878 (119885) + 120576 (8)where 120576 is the NN construction errors and satisfied 120576 le 120576119873and 120576119873 is a small bounded positive value

4 Complexity

3 Theoretical Developments

31 Adaptive Neural Control During the past two decadesvarious neural networks have been incorporated into adap-tive control for nonlinear systems with unknown dynamicsIn [42] a multiplayer discrete-time neural network con-troller was constructed for a class of multi-input multioutput(MIMO) dynamical systems where NNweights were trainedusing an improved online tuning algorithm An adaptiveNN output feedback control was proposed to control twoclasses of discrete-time systems in the presence of unknowncontrol directions [4] A robust adaptive neural controllerwas developed for a class of strict-feedback systems in[43] where a Nussbaum gain technique was employed todeal with unknown virtual control coefficients A dynamicrecurrent NN was employed for construction of an adaptiveobserver with online turned weights parameters in [44] andto deal with the time-delay of a class of nonlinear dynamicalsystems in [45] The time-delay of strict-feedback nonlinearsystems was also addressed by using NN control with properdesigned Lyapunov-Krasovskii functions in [46] For a classof unknown nonlinear affine time-delay systems an adaptivecontrol schemewas proposed by constructing two high-orderNNs for identifying system uncertainties [47] This idea hasbeen further extended to affine nonlinear systems with inputtime-delay in [48]

It should be noticed that piecewise continuous func-tions such as frictions backlash and dead-zone are widelyexisted in industrial plants Other than continuous nonlinearfunction the approximation of these piecewise functions ismore challenging since the NNrsquos universal approximationonly holds for continues functions To approximate thesepiecewise continuous functions a novel NN structure wasdesigned by involving a standard activation function and ajump approximation basis function [49] In [47] a CMACNN was employed for the closed-loop control of nonlineardynamical systems with rigorous stability analysis and in[50] a robust adaptive neural network control scheme wasdeveloped for cooperative tracking control of higher-ordernonlinear systems

The adaptive NN control scheme was also proposed forpure-feedback systems In [51] a high-order sliding modeobserverwas proposed to estimate the unknown system stateswhile two NNs were constructed to deal with approximationerrors and the unknown nonlinearities respectively In com-parison to the conventional control design for pure-feedbacksystems the state-feedback control was achieved withoutusing the backstepping technique In [52] a neural controlframework was proposed for nonlinear servo mechanism toguarantee both the steady-state and transient tracking per-formance In this work a prescribed performance functionwas employed in an output error transformation such thatthe tracking performance can be guaranteed by the regulationcontrol of the outputs In [53] an adaptive neural control wasalso designed for a class of nonlinear systems in the presenceof time-delays and input dead-zone and high-order neuralnetworks were employed to deal the unknown uncertaintiesIn this work a salient feature lies in the fact that only thenorm of the NNsrsquo weights (a scalar) needs to be online

updated such that the computational efficiency in the onlineimplementation could be significantly improved In [54] theauthors developed a neural network based feedforward con-trol to compensate for the nonlinearities and uncertaintiesof a dynamically substructured system consisting of bothnumerical and physical substructures where an adaptive lawwith a new leakage term ofNNweights error informationwasdeveloped to achieve improved convergence An experimenton a quasi-motorcycle testing rig validated the efficacy ofthis control strategy In [55] a neural dynamic control wasincorporated into the strict-feedback control of a class ofunknown nonlinear systems by using the dynamic surfacecontrol technique For a class of uncertain nonlinear systemswith unknown hysteresis NN was used for compensationof the nonlinearities [56] In [57] to deal with unknownnonsymmetrical input saturations of unknown nonaffinesystems NNs were used in the stateoutput feedback controlbased on the mean value theorem and the implicit functionTo avoid using the backstepping synthesis a dynamic surfacecontrol scheme was designed by combining the NN with anonlinear disturbance observer [58]

32 NN Based Adaptive Dynamic Programming In additionto adaptive control neural networks have also been adoptedto solve the optimization problem for nonlinear systemsIn convention optimal control the dynamic programmingmethod was widely used It aims to minimize a predefinedcost function such that a sequence of optimal control inputscould be derived However the cost function is usuallydifficult to online calculate due to the computation com-plexity in obtaining the solution of the Hamilton-Jacobi-Bellman (HJB) equationTherefore an adaptiveapproximatedynamic programming (ADP) technique was developed in[59] where a NN was trained to estimate the cost functionand then to derive solutions for the ADP Generally theADP has several different synonyms including approximatedynamic programming heuristic dynamic programming(HDP) critic network and reinforcement learning (RL) [60ndash62] Figure 4 shows the basic framework of the HDP with acritic-actor structure In [63] a discrete-time HJB equationwas solved using an NN based HDP algorithm to derive theoptimal control of nonlinear discrete-time systems In [64]three neural networks were constructed for an iterative ADPsuch that optimal feedback control of a discrete-time affinenonlinear system could be realized In [65] a globalized dualheuristic programming was presented to address the optimalcontrol of discrete-time systems In each iteration threeneural networks were used to learn the cost function andthe unknown nonlinear systems In [66] a reference networkcombining with an action network and a critic network wasintroduced in the ADP architecture to derive an internalgoal representation such that the learning and optimizationprocess could be facilitated The reference network has alsobeen introduced in the online action-dependent heuristicdynamic programming by employing a dual critic networkframework A policy iteration algorithm was introduced forinfinite horizon optimal control of nonlinear systems usingADP in [67] In [68] a reinforcement learning method wasintroduced for the stabilizing control of uncertain nonlinear

Complexity 5

Action network

Modelnetwork

Critic network

Critic network

x(k)u(k)

x(k + 1) U(k)

J(x(k))

J(x(k + 1))

minus

Figure 4 An overview of the HDP structure

systems in the presence of input constraints By using this RL-based controller a constrained optimal control problem wassolved with construction of only one critic neural network In[69] an ADP technique for online control and learning of ageneralized multiple-input-multiple-output (MIMO) systemwas investigated In [70] an adaptive NN based ADP controlscheme was presented for a class of nonlinear systems withunknown dynamics The optimal control law was calculatedby using a dual neural network scheme with a critic NN andan identifier NN Particularly parameters estimation errorwas used to online identify the learning weights to achievethe finite-time convergence Optimal tracking control for aclass of nonlinear systems was investigated in [71] where anew ldquoidentifier-criticrdquo based ADP framework was proposed

33 Evolutionary Computing In addition to the capacity ofapproximation and optimization of the NN there has beenalso a great interest in using the evolutionary approachesto train the neural networks With the evolvement of NNarchitectures learning rules connection weights and inputfeatures an evolutionary artificial neural network (EANN)was designed to provide superior performance in comparisonto conventional training approaches [72] A literature reviewof the EANNwas given in [73] where the evolution strategiessuch as feedforward artificial NN and genetic algorithms(GA) have been introduced for the EANNs In [72] severalEANN frameworks were introduced by embedding the evo-lution algorithms (EA) to evolve the NN structure In [74]an EPNet evolution system was proposed for evolving thefeedforward NN based on Fogelrsquos evolutionary programming(EP) method which could improve the NNrsquos connectionweights and architectures at the same time as well as decreasenoise in the fitness evaluation Good generalization abilityof the evolved NN has been constructed and verified inthe experiments In [75] a GA based technique has beenemployed to train the NNs in direct neural control systemssuch that the NN architectures could be optimized Adeficiency of the EANN is that the optimization processwould often result in a low training speed To overcomethis problem and facilitate adaptation processes a hybridmultiobjective evolutionary method was developed in [76]where the singular-value-decomposition (SVD) techniquewas employed to choose the necessary neurons number in thetraining of a feedforwardNNThe evolutionary approachwas

applied to identify a grey-box model with a multiobjectiveoptimization between the clearly known practical systemsand approximated nonlinear systems [77] Applications ofevolutionary algorithms for robotic navigation have beenintroduced and investigated in [78] A survey of machinelearning technique was reported in [79] where several meth-ods to improve the evolutionary computation were reviewed

The evolution algorithms have been employed in manyaspects for evolvements of NNs such as to train the NN con-nection weights or to obtain near-optimal NN architecturesas well as adapting learning rules of NNs to their environ-ment In a word the evolution algorithms provide NNs withthe ability of learning to learn and also to build the relation-ship between evolution and learning such that the EANNcould perform favorable ability to adapt to changes of thedynamic environment

4 Applications in Robots

41 NNBased RoboticManipulator Control Generally speak-ing the control methods for robot manipulators can beroughly divided into two groups model-free control andmodel based control For the model-free control approacheslike proportional-integral-derivative (PID) control satisfac-tory control performancemay not be guaranteed In contrastthe model based control approaches exhibit better controlbehavior but heavily depend on the validity of the robotmodel In practice however a perfect robotic dynamicmodel is always not available due to the complex mech-anisms and uncertainties Additionally the payload maybe varied according to different tasks which makes theaccurate dynamics model hard to be obtained in advance Tosolve such problems the NN approximation-based controlmethods have been used extensively in applications of robotmanipulator control A basic structure of the adaptive neuralnetwork control for robot manipulator is shown in Figure 5Consider a dynamic model of a robot manipulator given asfollows [80]

119872(119902) 119902 + 119862 ( 119902 119902) 119902 + 119866 (119902) = 120591 (9)

where119872(119902)119862( 119902 119902) and119866(119902) are the inertialmatrix Coriolismatrix and gravity vector respectively Then NN controldesign could be given as follows

120591119889 = minus11987011198901 minus 11987021198902 + 119878 (119885) (10)

6 Complexity

PD controller Robotic arm

Closed loop neural network controller

Neural network controller

Neuralinputs

qd qd e e q q q1

d

Figure 5 A basic structure of the adaptive NN robot control

where 1198901 is the tracking error 1198902 is the velocity tracking error119878(119885) is the NN controller with being the weights matrixand 119878(119885) being the NN regressor vector and 1198701 and 1198702 arecontrol gains specified by the designer

From (10) we can see that the robot controller consists ofa PD-like controller and aNN controller In traditionalmodelbased controllers the dynamic model of the robot could beregarded as a feedforward to address the effect caused bythe robot motion In practice however 119872(119902) 119862(119902 119902) and119866(119902) may not be known Therefore the NNs are used toapproximate the unknown dynamics 119891 = 119872(119902) 119902 +119862( 119902 119902) 119902 +119866(119902) and to improve the performance of the system via theonline estimation To adapt theNNweights adaptive laws aredesigned as follows

119882 = Γ (119878119890 minus 120590) (11)

where Γ and 120590 are specified positive parametersThe last termof right-hand side of (11) is the sigma modification whichis used to enhance the convergence and robustness of theparameters adaptation

In [80] a NN based share control method was devel-oped to control a teleoperated robot with environmentaluncertainties In this work the RBFNN was constructed tocompensate for the unknown dynamics of the teleoperatedrobot Particularly a shared control strategy was developedinto the controller to achieve the automatic obstacle avoid-ance combining with the information of visual camera andthe robot body such that the obstacle could be successfullyavoided and the operator could focus more on the operatedtask rather than the environment to guarantee the stabilityand manipulation In addition error transformations wereintegrated into the adaptiveNN control to guarantee the tran-sient control performance It was shown that by using theNNtechnique the control performance in both kinematic leveland dynamic level of the teleoperated robot was enhancedIn [81] an extreme learning machine (ELM) based controlstrategy was proposed for uncertain robot manipulators toidentify both the elasticity and geometry of an object ThisELM was applied to deal with the unknown nonlinearity ofthe robot manipulator to enhance the control performanceParticularly by utilizing ELM the proposed controller couldguarantee that the robot dynamics follow a reference modelsuch that the desired set point and the feedforward forcecould be updated to estimate the geometry and stiffness ofthe object As a result the reference model could be exactlymatched with a limited number of iterations

In [82] the NN controller was also employed to control awheel inverted pendulum which has been decomposed intotwo subsystems a fully actuated second-order planar movingsubsystem and a passive first-order pendulum subsystemThen the RBFNN was employed to compensate for theuncertain dynamics of the two subsystems by using itspowerful learning ability such that the enhanced controlperformance could be realized by using the NN learning In[83] a global adaptive neural control was proposed for a classof robot manipulators with finite-time convergence learn-ing performance This control scheme employed a smoothswitching mechanism combining with a nominal neuralnetwork controller and a robust controller to ensure globaluniform ultimately bounded stability The optimal weightswere obtained by the finite-time estimation algorithm suchthat after the learning process the learning weights could bereused next time for repeated tasks The global NN controlmechanism has been further extended to the control of dualarm robot manipulator in [84] where knowledge of bothrobot manipulator and the grasping object is unavailable inadvance By integrating prescribed functions into the designof controller the transient performance of the dual armrobot control was regularly guaranteed The NN was alsoemployed to deal with synchronization problem of multiplerobot manipulators in [85] where the reference trajectoriesare only available for part of the team members By usingthe NN approximation controller the robot has shown bettercontrol performance with enhanced transient performanceand enhanced robustness A RBFNN was constructed tocompensate for the nonlinear terms of a five-bar manipulatorbased on an error transformation function [86] The NNcontrol was also applied in the robot teleoperation control[87 88] Moreover a NN approximation technique wasemployed to deal with the unknown dynamics kinematicsand actuator properties in the manipulator tracking control[89]

42 NN Based Robot Control with Input NonlinearitiesAnother challenge of the robot manipulator is that theinput nonlinearities such as friction dead-zone and actuatorsaturation may inevitably exist in the robot systems Theseinput nonlinearities may lead to larger tracking errors anddegeneration of the control performance Therefore a num-ber of works have been proposed to handle the nonlinearitiesby utilizing the neural network design A neural adaptive

Complexity 7

controller was designed to deal with the effect of inputsaturation of the robot manipulator in [90] as follows

120591 = minus1199111 + 119863119878119863 (119885119863) 1205721 + 119862119878119862 (119885119862) 1205721+ 119866119878119866 (119885119866) + 119870119901 (1199112 + 120585) + 119870119903 sgn (1199112) (12)

where 1199111 is the robot position tracking error 1199112 is the velocitytracking error and 1205721 is an auxiliary controller 119863 119862 and119866 are the NN weights 119878119863(119885119863) 119878119862(119885119862) and 119878119866(119885119866) arethe NN regressor vectors and 119870119901 and 119870119903 are control gainsspecified by the designer 120585 is an auxiliary system designed toreduce the effect of the saturation with 120585 defined as follows

120585

= minus119870120585120585 minus

100381610038161003816100381610038161199111198792Δ12059110038161003816100381610038161003816 + (12) Δ120591119879Δ120591100381710038171003817100381712058510038171003817100381710038172 120585 + Δ120591 10038171003817100381710038171205851003817100381710038171003817 ge 1205830 10038171003817100381710038171205851003817100381710038171003817 lt 120583

(13)

where Δ120591 is the torque error caused by saturation and119870120585 is asmall positive value To update the NNweights adaptive lawsare designed as follows

119882119863 = Γ119863119896 (11987811986311989612057211198902119896 minus 120590119863119896119863119896)119882119862 = Γ119862119896 (11987811986211989612057211198902119896 minus 120590119862119896119862119896)119882119866 = Γ119866119896 (11987811986611989612057211198902119896 minus 120590119866119896119866119896)

(14)

where Γ119863119896 Γ119862119896 and Γ119866119896 are specified positive parameters and120590119863119896 120590119862119896 and 120590119866119896 are positive parametersIn [91] an adaptive neural network controller was con-

structed to approximate the input dead-zone and the uncer-tain dynamics of the robotic manipulator while the outputconstraint was also considered in the feedback control In[92] the NN was applied for the estimation of the unknownmodel parameters of a marine surface vessel and in [93] thefull-state constraint of an n-link robotic manipulator wasachieved by using the NN control The NN controller wasalso constructed for flexible roboticmanipulators to deal withthe vibration suppression based on a lumped spring-massmodel [94] while in [95] two RBFNNs were constructed forflexible robot manipulators to compensate for the unknowndynamics and the dead-zone effect respectively

The NN has also been used in many important industrialfields such as autonomous underwater vehicles (AUVs) andhypersonic flight vehicle (HFV) In [96] the NN has beenconstructed to deal with the attitude of AUVs in the presenceof input dead-zone and uncertain model parameters In [97]the adaptive neural control was employed to deal with under-water vehicle control in discrete-time domain encounteredwith the unknown input nonlinearities external disturbanceand model uncertainties Then the reinforcement learningwas applied to address these uncertainties by using a criticNN and an action NN The hypersonic flight vehicle controlwas investigated in [98] where the aerodynamic uncertaintiesand unknown disturbances were addressed by a disturbance

observer based NN In [99] a neural learning control wasembedded in the HFV controller to achieve the globalstability via a switching mechanism and a robust controller

43 NN Based Human-Robot Interaction Control Recentlythere is a predominant tendency to employ the robots inthe human-surrounded environment such as householdservices or industrial applications where humans and robotsmay interact with each other directly Therefore interactioncontrol has become a promising research field and has beenwidely studied In [100] a learning method was developedsuch that the dynamics of a robot arm could follow atarget impedance model with only knowledge of the roboticstructure (see Figure 6)

TheNNwas further employed in robot control in interac-tionwith an environment [101] where impedance control wasachieved with the completely unknown robotic dynamicsIn [102] a learning method was developed such that therobot was able to adjust the impedance parameters when itinteracted with unknown environments In order to learnoptimal impedance parameters in the robot manipulatorcontrol an adaptive dynamic programming (ADP) methodwas employed when the robot interacted with unknowntime-varying environments where NNs were used for bothcritic and actor networks [103] The ADP was also employedfor coordination of multirobots [104] in which possibledisagreement between different manipulators was handledand dynamics of both robots and themanipulated object werenot required to be known

In this work the controller consists of two parts a criticnetwork which was used to approximate the cost functionand an actual NN which was designed to control the robotThe critic NN is designed as follows [104]

Υ (119905) = 119862119878119862 (119885119862) (15)

where 119885119862 = 120585 120585 = [119879119900 119911119879 119909119879119900 ]119879 with 119909119900 being the positionof the object and 119911 being the tracking error 119862 is the NNweight and 119878119862 is the regressor vector

The critic NN is used to approximate a cost function119888(119905) = 120585119879119876120585 + 119906119879119877119906 where 119906 denotes the control inputand 119876 and 119877 are positive definite matrix Since the controlobjective is to minimize the control effort the adaptation lawis designed as

119882119888 = minus120590119888 120597119864119888120597119888 = 120590119888 (119888 minus 119879119888 119878119862) 119878119862 (16)

where 120590119888 is the learning rate and 119864119888 = (12)(119888 minus 119879119888 119878119862)2On the other hand the actual NN control is designed to

control the robot as

119906 = 119879119886 119878119886 (119885119886) minus 119890 minus 1198702119890V (17)

where 119879119886 119878119886(119885119886) could learn the dynamics of the robotwith 119886 being the NN weight and 119878119886 being the regressorvector 119890 and 119890V are the position and velocity tracking errorsrespectively and 1198702 is the control gain

8 Complexity

Positon control Robotic arm Inverse

kinematics

Forward kinematics

Optimal critic learning

EnvironmentImpedance

model

xd

xrqr

x q

f

minus

+

Figure 6 Block of the learning impedence control

Since the control objective is to guarantee the estimationof both robot dynamics and the cost function Υ(119905) theadaptive law is selected as follows

119882119879119886119894 = minus120590119886 (119879119886119894119878119886 + 119896119903Υ) 119878119886 (18)

where 120590119886 and 119896119903 are positive constantsOn the other hand as a fundamental element of the next-

generation robots the human-robot collaboration (HRC) hasbeen widely studied by roboticists and NN is employed inHRCwith its powerful learning ability In [105] theNNswereemployed to estimate the human partnerrsquos motion intentionin human-robot collaboration such that the robot was able toactively follow its human partner To adjust the robotrsquos role tolead or to follow according to the humanrsquos intention gametheory was employed for fundamental analysis of human-robot interaction and an adaptation law was developed in[106] Policy iteration combining with NN was adopted toprovide a rigorous solution to the problem of the systemequilibrium in human-robot interaction [107]

44 NN Based Robot Cognitive Control According to thepredictive processing theory [108] the human brain is alwaysactively anticipating the incoming sensorimotor informationThis process exists because the living beings exhibit latenciesdue to neural processing delays and a limited bandwidthin their sensorimotor processing To compensate for sucha delay in human brain neural feedback signals (includinglateral and top-down connections)modulate the neural activ-ities via inhibitory or excitatory connections by influencingthe neuronal population coding of the bottom-up sensory-driven signals in the perception-action system Similarlyin robotic systems it is claimed that such a delay and alimited bandwidth also can be compensated by the predictivefunctions learnt by recurrent neural models Such a learningprocess can be done via only visual processing [109] or in theloop of perception and action [110]

Based on the hierarchical sensorimotor integration the-ory which advocates that action and perception are inter-twined by sharing the same representational basis [111] therepresentation on different levels of sensory perception doesnot explicitly represent actions instead there is an encodingof the possible future percept which is learnt from priorsensorimotor knowledge

In the Bayesian once this perception and action linkshave been established after learning these perception-action

associations in this architecture allow the following opera-tions

First these associations allow predicting the perceptualoutcome of given actions by means of the forward models(eg Bayesian model) It can be written as

119875 (119864 | 119860 119868) prop 119875 (119860 | 119864) 119875 (119864 | 119868) (19)

where E estimates the upcoming perception evidence givenan executed action A and other prior information you havealready known in 119868 The term 119875(119864 | 119860 119868) suggests that aprelearntmodel representing the possibility of amotor actionA will be executed given that a (possible) resulting sensoryevidence 119864 is perceived (backward computation)

Second these associations allow selecting an appropri-ate movement given an intended perceptual representationFrom the backward computations introduced in the followingequation a predictive sensorimotor integration occurs

119875 (119860 | 119864 119866) prop 119875 (119864 | 119860) 119875 (119860 | 119866) (20)

where A indicates a particular action selected given the(intended) sensory information 119864 and a goal G Here weassume that onersquos action is only determined by the currentsensory input and the goal

In terms of its hierarchical organization it also allows thisoperation with bidirectional information pathways a lowlevel perception representation can be expressed on a higherlevel with a more complex receptive field and vice versa119890low hArr 119890high This can be realized by the bidirectional deeparchitectures such as [112] Conceptually these operationscan be achieved by extracting statistical regularity shown inFigure 7

Since both perception and action processes can be seenas temporal sequences from the mathematical perspectivethe recurrent networks are Turing-Complete and have alearning capacity to learn time sequences with arbitrarylength [113] if properly trained Furthermore such recurrentconnections can be placed in a hierarchical way in whichthe prediction functions on different layers attempt to predictthe nonlinear time-series in different time-scales [114] Fromthis point the recurrent neural network with parametric biasunits (RNNPB) [115] andmultiple time-scale recurrent neuralnetworks (MTRNN) [116] were applied to predict sequencesby understanding them in various temporal levels

The difference of the temporal levels controls the prop-erties of the different levels of the presentation in the deep

Complexity 9

SMA

M1

Brainstem

Spine

Action

Common coding

Common coding

MST

V5MT

V3

V1

Cognitiveprocesses

Visual perception(dorsal)

Figure 7 The process of cognitive control

recurrent network For instance in the MTRNN network[112] the learning of each neuron follows the updating rule ofclassical firing rate models in which the activity of a neuronis determined by the average firing rate of all the connectedneurons Additionally the neuronal activity is also decayingover time following an updating rule of leaky integratormodel Assuming the i-th MTRNN neuron has the numberof N connections the current membrane potential status ofa neuron can be defined by both the previous activation andthe current synaptic inputs

119906119894119905+1 = (1 minus 1120591119894)119906119894119905 + 1120591119894 [sum119908119894119895119909119895119905] (21)

where 119908119894119895 represents the synaptic weight from the j-thneuron to the i-th neuron 119909119895119905 is the activity of j-th neuronat t-th time-step and 120591 is the time-scale parameter whichdetermines the decay rate of this neuron a larger 120591 meanstheir activities change slowly over time compared with thosewith a smaller time-scale parameter 120591

In [117] the concepts of predictive coding were discussedin detail where the learning generation and recognition ofactions can be conducted by means of the principle of pre-diction error minimization By using the predictive codingthe RNNPB and MTRNN are capable for both generatingown actions and recognizing the same actions performedby others Recently the study on neurorobotics experimentshas shown that the dynamic predictive coding scheme canbe used to address fluctuations in temporal patterns whentraining a recurrent neural network (RNN) model [118]This predictive coding scheme enables organisms to predictperceptual outcomes based on current intentions of actionsto the external environment and to forecast perceptualsequences corresponding to given intention states [118]

Based on this architecture two-layer RNN models wereutilized to extract visual information [119] and to under-stand intentions [120] or emotion status [121] in socialrobotics three-layer RNNmodels were used to integrate and

understand multimodal information for a humanoid iCubrobot [112 122] Moreover the predictive coding frameworkhas been extended to variational Bayes predictive codingMTRNN which can arbitrate between deterministic modeland probabilistic model by setting a metaparameter [123]Such extension could provide significant improvement indealing with noisy fluctuated sensory inputs which robots areexpected to experience in more real world setting In [124]a MTRNN was employed to control a humanoid robot andexperimental results have shown that by using only partialtraining data the control model can achieve generalizationby learning in a lower feature perception level

The hierarchical structure of RNN exhibits a greatlearning capacity to store multimodal information whichis beneficial for the robotic systems to understand and topredict in a complex environment As the future models andapplications the state-of-the-art deep learning techniques orthemotor actions of robotic systems can be further integratedinto this predictive architecture

5 Conclusion

In summary great achievements for control design of nonlin-ear system by means of neural networks have been gained inthe last two decades Despite the impossibility in identifyingor listing all the related contributions in this short reviewefforts have been made to summarize the recent progressin the area of NN control and its particular applications inthe robot learning control the robot interaction control andthe robot recognition control In this paper we have shownthat significant progress of NN has been made in control ofthe nonlinear systems in solving the optimization problemin approximating the system dynamics in dealing with theinput nonlinearities in human-robot interaction and in thepattern recognition All these developments accompany notonly the development of techniques in control and advancedmanufactures but also theatrical progress in constructingand developing the neural networks Although huge efforts

10 Complexity

have been made to embed the NN in practical control sys-tems there is still a large gap between the theory and practiceTo improve the feasibility and usability the evolutionarycomputing theory has been proposed to train the NNs It canautomatically find a near-optimal NN architecture and allowa NN to adapt its learning rule to its environment Howeverthe complex and long training process of the evolutionaryalgorithms deters their practical applications More effortsneed to be made to evolve the NN architecture and NNlearning technique in the control design On the otherhand in human brain the neural activities are modulatedvia inhibitory or excitatory connections by influencing theneuronal population coding of the bottom-up sensory-drivensignals in the perception-action system In this sense howto integrate the sensor-motor information into the networkto make NNs more feasible to adapt to the environment andto resemble the capacity of the human brain deserves furtherinvestigations

In conclusion a brief review on neural networks for thecomplex nonlinear systems is provided with adaptive neuralcontrol NN based dynamic programming evolution com-puting and their practical applications in the robotic fieldsWe believe this area may promote increasing investigationsin both theories and applications And emerging topics likedeep learning [125ndash128] big data [129ndash131] and cloud com-puting may be incorporated into the neural network controlfor complex systems for example deep neural networkscould be used to process massive amounts of unsuperviseddata in complex scenarios neural networks can be helpfulin reducing the data dimensionality and the optimization ofNN training may be employed to enhance the learning andadaptation performance of robots

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

Thisworkwas partially supported by theNational Nature Sci-ence Foundation (NSFC) under Grant 61473120 GuangdongProvincial Natural Science Foundation 2014A030313266International Science and Technology Collaboration Grant2015A050502017 Science andTechnology Planning Project ofGuangzhou 201607010006 State Key Laboratory of Roboticsand System (HIT) Grant SKLRS-2017-KF-13 and the Funda-mental Research Funds for the Central Universities

References

[1] F Gruau D Whitley and L Pyeatt ldquoA comparison betweencellular encoding and direct encoding for genetic neural net-worksrdquo inProceedings of the 1st annual conference on genetic pro-gramming pp 81ndash89 1996

[2] H de Garis ldquoAn artificial brain ATRrsquos CAM-Brain Project aimsto buildevolve an artificial brain with a million neural netmodules inside a trillion cell Cellular Automata Machinerdquo NewGeneration Computing vol 12 no 2 pp 215ndash221 1994

[3] W S McCulloch and W Pitts ldquoA logical calculus of the ideasimmanent in nervous activityrdquoBulletin ofMathematical Biologyvol 5 no 4 pp 115ndash133 1943

[4] C Yang S S Ge C Xiang T Chai and T H Lee ldquoOutput feed-back NN control for two classes of discrete-time systems withunknown control directions in a unified approachrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 19 no 11pp 1873ndash1886 2008

[5] R Kumar R K Aggarwal and J D Sharma ldquoEnergy analysis ofa building using artificial neural network A reviewrdquo Energy andBuildings vol 65 pp 352ndash358 2013

[6] C Alippi C De Russis and V Piuri ldquoA neural-network basedcontrol solution to air-fuel ratio control for automotive fuel-injection systemsrdquo IEEE Transactions on Systems Man andCybernetics Part C Applications and Reviews vol 33 no 2 pp259ndash268 2003

[7] E M Azoff Neural network time series forecasting of financialmarkets John Wiley amp Sons Inc 1994

[8] I Fister P N Suganthan J Fister et al ldquoArtificial neural net-work regression as a local search heuristic for ensemble strate-gies in differential evolutionrdquo Nonlinear Dynamics vol 84 no2 pp 895ndash914 2016

[9] Y Zhang S Li and H Guo ldquoA type of biased consensus-based distributed neural network for path planningrdquoNonlinearDynamics vol 89 no 3 pp 1803ndash1815 2017

[10] X Shi Z Wang and L Han ldquoFinite-time stochastic synchro-nization of time-delay neural networks with noise disturbancerdquoNonlinear Dynamics vol 88 no 4 pp 2747ndash2755 2017

[11] P He and Y Li ldquoHinfin synchronization of coupled reaction-diffusion neural networks with mixed delaysrdquo Complexity vol21 no S2 pp 42ndash53 2016

[12] Z Tu J Cao A Alsaedi F E Alsaadi and T Hayat ldquoGlobalLagrange stability of complex-valued neural networks of neutraltype with time-varying delaysrdquo Complexity vol 21 no S2 pp438ndash450 2016

[13] C Wang S Guo and Y Xu ldquoFormation of autapse connectedto neuron and its biological functionrdquo Complexity vol 2017Article ID 5436737 9 pages 2017

[14] J D J Rubio I Elias D R Cruz and J Pacheco ldquoUniform stableradial basis function neural network for the prediction in twomechatronic processesrdquo Neurocomputing vol 227 pp 122ndash1302017

[15] J d Rubio ldquoUSNFIS Uniform stable neuro fuzzy inferencesystemrdquo Neurocomputing vol 262 pp 57ndash66 2017

[16] Q Liu J Yin V C M Leung J-H Zhai Z Cai and J LinldquoApplying a new localized generalization error model to designneural networks trained with extreme learning machinerdquo Neu-ral Computing and Applications pp 1ndash8 2014

[17] R J de Jesus ldquoInterpolation neural network model of a manu-factured wind turbinerdquoNeural Computing and Applications pp1ndash12 2016

[18] C Mu and D Wang ldquoNeural-network-based adaptive guaran-teed cost control of nonlinear dynamical systems with matcheduncertaintiesrdquo Neurocomputing vol 245 pp 46ndash54 2017

[19] Z Lin DMa J Meng and L Chen ldquoRelative ordering learningin spiking neural network for pattern recognitionrdquo Neuro-computing 2017

[20] J Yu J Sang and X Gao ldquoMachine learning and signal pro-cessing for big multimedia analysisrdquo Neurocomputing vol 257pp 1ndash4 2017

Complexity 11

[21] T Zhang S S Ge and C C Hang ldquoAdaptive neural networkcontrol for strict-feedback nonlinear systems using backstep-ping designrdquo Automatica vol 36 no 12 pp 1835ndash1846 2000

[22] S S Ge and T Zhang ldquoNeural-network control of nonaffinenonlinear system with zero dynamics by state and outputfeedbackrdquo IEEE Transactions on Neural Networks and LearningSystems vol 14 no 4 pp 900ndash918 2003

[23] S S Ge C Yang and T H Lee ldquoAdaptive predictive controlusing neural network for a class of pure-feedback systems in dis-crete timerdquo IEEE Transactions on Neural Networks and LearningSystems vol 19 no 9 pp 1599ndash1614 2008

[24] F W Lewis S Jagannathan and A Yesildirak Neural networkcontrol of robotmanipulators and non-linear systems CRCPress1998

[25] S Jagannathan and F L Lewis ldquoIdentification of nonlineardynamical systems using multilayered neural networksrdquo Auto-matica vol 32 no 12 pp 1707ndash1712 1996

[26] D Vrabie and F Lewis ldquoNeural network approach tocontinuous-time direct adaptive optimal control for partiallyunknown nonlinear systemsrdquo Neural Networks vol 22 no 3pp 237ndash246 2009

[27] C Yang S S Ge and T H Lee ldquoOutput feedback adaptive con-trol of a class of nonlinear discrete-time systems with unknowncontrol directionsrdquo Automatica vol 45 no 1 pp 270ndash2762009

[28] C Yang Z Li and J Li ldquoTrajectory planning and optimizedadaptive control for a class of wheeled inverted pendulumvehicle modelsrdquo IEEE Transactions on Cybernetics vol 43 no 1pp 24ndash36 2013

[29] Y Jiang C Yang and H Ma ldquoA review of fuzzy logic andneural network based intelligent control design for discrete-time systemsrdquo Discrete Dynamics in Nature and Society ArticleID 7217364 Art ID 7217364 11 pages 2016

[30] Y Jiang C Yang S-L Dai and B Ren ldquoDeterministic learningenhanced neutral network control of unmanned helicopterrdquoInternational Journal of Advanced Robotic Systems vol 13 no6 pp 1ndash12 2016

[31] Y Jiang Z Liu C Chen and Y Zhang ldquoAdaptive robust fuzzycontrol for dual arm robot with unknown input deadzonenonlinearityrdquo Nonlinear Dynamics vol 81 no 3 pp 1301ndash13142015

[32] httpsstemcellthailandorgneurons-definition-function-neurotransmitters

[33] MDefoort T Floquet A Kokosy andW Perruquetti ldquoSliding-mode formation control for cooperative autonomous mobilerobotsrdquo IEEE Transactions on Industrial Electronics vol 55 no11 pp 3944ndash3953 2008

[34] X Liu C Yang Z Chen MWang and C Su ldquoNeuro-adaptiveobserver based control of flexible joint robotrdquoNeurocomputing2017

[35] F Hamerlain T Floquet and W Perruquetti ldquoExperimentaltests of a slidingmode controller for trajectory tracking of a car-like mobile robotrdquo Robotica vol 32 no 1 pp 63ndash76 2014

[36] R J de Jesus ldquoDiscrete time control based in neural networksfor pendulumsrdquo Applied Soft Computing 2017

[37] Y Pan M J Er T Sun B Xu and H Yu ldquoAdaptive fuzzy PDcontrol with stable Hinfin tracking guaranteerdquo Neurocomputingvol 237 pp 71ndash78 2017

[38] R J de Jesus ldquoAdaptive least square control in discrete time ofrobotic armsrdquo Soft Computing vol 19 no 12 pp 3665ndash36762015

[39] S Commuri S Jagannathan and F L Lewis ldquoCMAC neuralnetwork control of robot manipulatorsrdquo Journal of RoboticSystems vol 14 no 6 pp 465ndash482 1997

[40] J S Albus ldquoTheoretical and experimental aspects of a cerebellarmodelrdquoDevelopmental Disabilities Research Reviews vol 17 pp93ndash101 1972

[41] B Yang R Bao and H Han ldquoRobust hybrid control based onPD and novel CMAC with improved architecture and learningscheme for electric load simulatorrdquo IEEE Transactions onIndustrial Electronics vol 61 no 10 pp 5271ndash5279 2014

[42] S Jagannathan and F L Lewis ldquoMultilayer discrete-timeneural-net controller with guaranteed performancerdquo IEEETransactions on Neural Networks and Learning Systems vol 7no 1 pp 107ndash130 1996

[43] S S Ge and J Wang ldquoRobust adaptive neural control for a classof perturbed strict feedback nonlinear systemsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no6 pp 1409ndash1419 2002

[44] Y H Kim F L Lewis and C T Abdallah ldquoA dynamic recurrentneural-network-based adaptive observer for a class of nonlinearsystemsrdquo Automatica vol 33 no 8 pp 1539ndash1543 1997

[45] J-Q Huang and F L Lewis ldquoNeural-network predictive controlfor nonlinear dynamic systems with time-delayrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 14 no 2pp 377ndash389 2003

[46] S S Ge F Hong and T H Lee ldquoAdaptive neural control ofnonlinear time-delay systems with unknown virtual controlcoefficientsrdquo IEEE Transactions on Systems Man and Cybernet-ics Part B Cybernetics vol 34 no 1 pp 499ndash516 2004

[47] J Na X M Ren and H Huang ldquoTime-delay positive feedbackcontrol for nonlinear time-delay systems with neural networkcompensationrdquo Zidonghua XuebaoActa Automatica Sinica vol34 no 9 pp 1196ndash1202 2008

[48] J Na X Ren C Shang and Y Guo ldquoAdaptive neural networkpredictive control for nonlinear pure feedback systems withinput delayrdquo Journal of Process Control vol 22 no 1 pp 194ndash206 2012

[49] R R Selmic and F L Lewis ldquoNeural-network approximationof piecewise continuous functions application to friction com-pensationrdquo IEEE Transactions on Neural Networks and LearningSystems vol 13 no 3 pp 745ndash751 2002

[50] H Zhang and F L Lewis ldquoAdaptive cooperative trackingcontrol of higher-order nonlinear systems with unknowndynamicsrdquo Automatica vol 48 no 7 pp 1432ndash1439 2012

[51] J Na X Ren and D Zheng ldquoAdaptive control for nonlinearpure-feedback systems with high-order sliding mode observerrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 24 no 3 pp 370ndash382 2013

[52] J Na Q Chen X Ren and Y Guo ldquoAdaptive prescribedperformancemotion control of servomechanisms with frictioncompensationrdquo IEEE Transactions on Industrial Electronics vol61 no 1 pp 486ndash494 2014

[53] J Na X Ren G Herrmann and Z Qiao ldquoAdaptive neuraldynamic surface control for servo systems with unknown dead-zonerdquoControl Engineering Practice vol 19 no 11 pp 1328ndash13432011

[54] G Li J Na D P Stoten and X Ren ldquoAdaptive neural networkfeedforward control for dynamically substructured systemsrdquoIEEE Transactions on Control Systems Technology vol 22 no3 pp 944ndash954 2014

Complexity 3

Hidden layer

Output layerInput layer

Σ

Σ

z1

z2

zm

w11

w21

w31w2n w1n

w3n

wl1wln

f1(z)

fn(z)

Figure 2 Structure of the RBFNN

z1

zm

Input space

Association memory space

Weight memory space

Output space

Receptive field space

Σ

Σ

f1(z)

fn(z)

Figure 3 Structure of a CMAC neural network

space As shown in Figure 3 two components are involvedin the CMAC neural network to determine the value of theapproximated nonlinear function 119866(119885)

119877 119885 997888rarr 119862119875 119862 997888rarr 119865 (4)

where

119885 ism-dimensional input space

F is n-dimensional output space

C is119873119888-dimensional association space

and 119877(sdot) denotes the mapping from the input vector tothe association space that is 120572 = 119877(119885) The outputsare computed through 119875(120572) by using a projection of theassociation vector 120572 onto a weights vector such that

119891 = 119875 (120572) = 119882119879120572 (5)

It should be noted that 119877(119885) can be represented by amultidimensional receptive filed function such that eachpoint in input 119885 is assigned with an activation value The

receptive-field basis functions of the association vector couldbe chosen as Gaussian functions as follows

ℎ119894119896 (1003817100381710038171003817119911119894 minus 1199061198941198961003817100381710038171003817) = exp[minus (119911119894 minus 119906119894119896)21205992119894119896 ] 119896 = 1 2 119897

(6)

where l is number of blocks of the associate space ℎ119894119896denotes the kth block associated with the input 119911119894 119906119894119896denotes the receptive fieldrsquos center and 120599119894119896 is the varianceof Gaussian function Then the multidimensional receptive-field function can be described as

119878 (119885) = [1199041 1199042 119904119899]119879 (7)

where 119904119896(119885 119906119896 120599119896) = prod119898119894=1ℎ119894119896(119911119894) 119906119896 = [1199061119896 1199062119896 119906119898119896]119879and 120599119896 = [1205991119896 1205992119896 120599119898119896]119879 The following property showsthe approximation ability provided by the CMAC neuralnetwork

Lemma 1 For a continuous nonlinear function 119865(119885) thereexists an ideal weight value 119882lowast such that 119865(119885) could beuniformly approximated by a CMAC with the multiplicationof the optimal weights119882lowast and the associate vector 119878(119885) as

119865 (119885) = 119882119879119878 (119885) + 120576 (8)where 120576 is the NN construction errors and satisfied 120576 le 120576119873and 120576119873 is a small bounded positive value

4 Complexity

3 Theoretical Developments

31 Adaptive Neural Control During the past two decadesvarious neural networks have been incorporated into adap-tive control for nonlinear systems with unknown dynamicsIn [42] a multiplayer discrete-time neural network con-troller was constructed for a class of multi-input multioutput(MIMO) dynamical systems where NNweights were trainedusing an improved online tuning algorithm An adaptiveNN output feedback control was proposed to control twoclasses of discrete-time systems in the presence of unknowncontrol directions [4] A robust adaptive neural controllerwas developed for a class of strict-feedback systems in[43] where a Nussbaum gain technique was employed todeal with unknown virtual control coefficients A dynamicrecurrent NN was employed for construction of an adaptiveobserver with online turned weights parameters in [44] andto deal with the time-delay of a class of nonlinear dynamicalsystems in [45] The time-delay of strict-feedback nonlinearsystems was also addressed by using NN control with properdesigned Lyapunov-Krasovskii functions in [46] For a classof unknown nonlinear affine time-delay systems an adaptivecontrol schemewas proposed by constructing two high-orderNNs for identifying system uncertainties [47] This idea hasbeen further extended to affine nonlinear systems with inputtime-delay in [48]

It should be noticed that piecewise continuous func-tions such as frictions backlash and dead-zone are widelyexisted in industrial plants Other than continuous nonlinearfunction the approximation of these piecewise functions ismore challenging since the NNrsquos universal approximationonly holds for continues functions To approximate thesepiecewise continuous functions a novel NN structure wasdesigned by involving a standard activation function and ajump approximation basis function [49] In [47] a CMACNN was employed for the closed-loop control of nonlineardynamical systems with rigorous stability analysis and in[50] a robust adaptive neural network control scheme wasdeveloped for cooperative tracking control of higher-ordernonlinear systems

The adaptive NN control scheme was also proposed forpure-feedback systems In [51] a high-order sliding modeobserverwas proposed to estimate the unknown system stateswhile two NNs were constructed to deal with approximationerrors and the unknown nonlinearities respectively In com-parison to the conventional control design for pure-feedbacksystems the state-feedback control was achieved withoutusing the backstepping technique In [52] a neural controlframework was proposed for nonlinear servo mechanism toguarantee both the steady-state and transient tracking per-formance In this work a prescribed performance functionwas employed in an output error transformation such thatthe tracking performance can be guaranteed by the regulationcontrol of the outputs In [53] an adaptive neural control wasalso designed for a class of nonlinear systems in the presenceof time-delays and input dead-zone and high-order neuralnetworks were employed to deal the unknown uncertaintiesIn this work a salient feature lies in the fact that only thenorm of the NNsrsquo weights (a scalar) needs to be online

updated such that the computational efficiency in the onlineimplementation could be significantly improved In [54] theauthors developed a neural network based feedforward con-trol to compensate for the nonlinearities and uncertaintiesof a dynamically substructured system consisting of bothnumerical and physical substructures where an adaptive lawwith a new leakage term ofNNweights error informationwasdeveloped to achieve improved convergence An experimenton a quasi-motorcycle testing rig validated the efficacy ofthis control strategy In [55] a neural dynamic control wasincorporated into the strict-feedback control of a class ofunknown nonlinear systems by using the dynamic surfacecontrol technique For a class of uncertain nonlinear systemswith unknown hysteresis NN was used for compensationof the nonlinearities [56] In [57] to deal with unknownnonsymmetrical input saturations of unknown nonaffinesystems NNs were used in the stateoutput feedback controlbased on the mean value theorem and the implicit functionTo avoid using the backstepping synthesis a dynamic surfacecontrol scheme was designed by combining the NN with anonlinear disturbance observer [58]

32 NN Based Adaptive Dynamic Programming In additionto adaptive control neural networks have also been adoptedto solve the optimization problem for nonlinear systemsIn convention optimal control the dynamic programmingmethod was widely used It aims to minimize a predefinedcost function such that a sequence of optimal control inputscould be derived However the cost function is usuallydifficult to online calculate due to the computation com-plexity in obtaining the solution of the Hamilton-Jacobi-Bellman (HJB) equationTherefore an adaptiveapproximatedynamic programming (ADP) technique was developed in[59] where a NN was trained to estimate the cost functionand then to derive solutions for the ADP Generally theADP has several different synonyms including approximatedynamic programming heuristic dynamic programming(HDP) critic network and reinforcement learning (RL) [60ndash62] Figure 4 shows the basic framework of the HDP with acritic-actor structure In [63] a discrete-time HJB equationwas solved using an NN based HDP algorithm to derive theoptimal control of nonlinear discrete-time systems In [64]three neural networks were constructed for an iterative ADPsuch that optimal feedback control of a discrete-time affinenonlinear system could be realized In [65] a globalized dualheuristic programming was presented to address the optimalcontrol of discrete-time systems In each iteration threeneural networks were used to learn the cost function andthe unknown nonlinear systems In [66] a reference networkcombining with an action network and a critic network wasintroduced in the ADP architecture to derive an internalgoal representation such that the learning and optimizationprocess could be facilitated The reference network has alsobeen introduced in the online action-dependent heuristicdynamic programming by employing a dual critic networkframework A policy iteration algorithm was introduced forinfinite horizon optimal control of nonlinear systems usingADP in [67] In [68] a reinforcement learning method wasintroduced for the stabilizing control of uncertain nonlinear

Complexity 5

Action network

Modelnetwork

Critic network

Critic network

x(k)u(k)

x(k + 1) U(k)

J(x(k))

J(x(k + 1))

minus

Figure 4 An overview of the HDP structure

systems in the presence of input constraints By using this RL-based controller a constrained optimal control problem wassolved with construction of only one critic neural network In[69] an ADP technique for online control and learning of ageneralized multiple-input-multiple-output (MIMO) systemwas investigated In [70] an adaptive NN based ADP controlscheme was presented for a class of nonlinear systems withunknown dynamics The optimal control law was calculatedby using a dual neural network scheme with a critic NN andan identifier NN Particularly parameters estimation errorwas used to online identify the learning weights to achievethe finite-time convergence Optimal tracking control for aclass of nonlinear systems was investigated in [71] where anew ldquoidentifier-criticrdquo based ADP framework was proposed

33 Evolutionary Computing In addition to the capacity ofapproximation and optimization of the NN there has beenalso a great interest in using the evolutionary approachesto train the neural networks With the evolvement of NNarchitectures learning rules connection weights and inputfeatures an evolutionary artificial neural network (EANN)was designed to provide superior performance in comparisonto conventional training approaches [72] A literature reviewof the EANNwas given in [73] where the evolution strategiessuch as feedforward artificial NN and genetic algorithms(GA) have been introduced for the EANNs In [72] severalEANN frameworks were introduced by embedding the evo-lution algorithms (EA) to evolve the NN structure In [74]an EPNet evolution system was proposed for evolving thefeedforward NN based on Fogelrsquos evolutionary programming(EP) method which could improve the NNrsquos connectionweights and architectures at the same time as well as decreasenoise in the fitness evaluation Good generalization abilityof the evolved NN has been constructed and verified inthe experiments In [75] a GA based technique has beenemployed to train the NNs in direct neural control systemssuch that the NN architectures could be optimized Adeficiency of the EANN is that the optimization processwould often result in a low training speed To overcomethis problem and facilitate adaptation processes a hybridmultiobjective evolutionary method was developed in [76]where the singular-value-decomposition (SVD) techniquewas employed to choose the necessary neurons number in thetraining of a feedforwardNNThe evolutionary approachwas

applied to identify a grey-box model with a multiobjectiveoptimization between the clearly known practical systemsand approximated nonlinear systems [77] Applications ofevolutionary algorithms for robotic navigation have beenintroduced and investigated in [78] A survey of machinelearning technique was reported in [79] where several meth-ods to improve the evolutionary computation were reviewed

The evolution algorithms have been employed in manyaspects for evolvements of NNs such as to train the NN con-nection weights or to obtain near-optimal NN architecturesas well as adapting learning rules of NNs to their environ-ment In a word the evolution algorithms provide NNs withthe ability of learning to learn and also to build the relation-ship between evolution and learning such that the EANNcould perform favorable ability to adapt to changes of thedynamic environment

4 Applications in Robots

41 NNBased RoboticManipulator Control Generally speak-ing the control methods for robot manipulators can beroughly divided into two groups model-free control andmodel based control For the model-free control approacheslike proportional-integral-derivative (PID) control satisfac-tory control performancemay not be guaranteed In contrastthe model based control approaches exhibit better controlbehavior but heavily depend on the validity of the robotmodel In practice however a perfect robotic dynamicmodel is always not available due to the complex mech-anisms and uncertainties Additionally the payload maybe varied according to different tasks which makes theaccurate dynamics model hard to be obtained in advance Tosolve such problems the NN approximation-based controlmethods have been used extensively in applications of robotmanipulator control A basic structure of the adaptive neuralnetwork control for robot manipulator is shown in Figure 5Consider a dynamic model of a robot manipulator given asfollows [80]

119872(119902) 119902 + 119862 ( 119902 119902) 119902 + 119866 (119902) = 120591 (9)

where119872(119902)119862( 119902 119902) and119866(119902) are the inertialmatrix Coriolismatrix and gravity vector respectively Then NN controldesign could be given as follows

120591119889 = minus11987011198901 minus 11987021198902 + 119878 (119885) (10)

6 Complexity

PD controller Robotic arm

Closed loop neural network controller

Neural network controller

Neuralinputs

qd qd e e q q q1

d

Figure 5 A basic structure of the adaptive NN robot control

where 1198901 is the tracking error 1198902 is the velocity tracking error119878(119885) is the NN controller with being the weights matrixand 119878(119885) being the NN regressor vector and 1198701 and 1198702 arecontrol gains specified by the designer

From (10) we can see that the robot controller consists ofa PD-like controller and aNN controller In traditionalmodelbased controllers the dynamic model of the robot could beregarded as a feedforward to address the effect caused bythe robot motion In practice however 119872(119902) 119862(119902 119902) and119866(119902) may not be known Therefore the NNs are used toapproximate the unknown dynamics 119891 = 119872(119902) 119902 +119862( 119902 119902) 119902 +119866(119902) and to improve the performance of the system via theonline estimation To adapt theNNweights adaptive laws aredesigned as follows

119882 = Γ (119878119890 minus 120590) (11)

where Γ and 120590 are specified positive parametersThe last termof right-hand side of (11) is the sigma modification whichis used to enhance the convergence and robustness of theparameters adaptation

In [80] a NN based share control method was devel-oped to control a teleoperated robot with environmentaluncertainties In this work the RBFNN was constructed tocompensate for the unknown dynamics of the teleoperatedrobot Particularly a shared control strategy was developedinto the controller to achieve the automatic obstacle avoid-ance combining with the information of visual camera andthe robot body such that the obstacle could be successfullyavoided and the operator could focus more on the operatedtask rather than the environment to guarantee the stabilityand manipulation In addition error transformations wereintegrated into the adaptiveNN control to guarantee the tran-sient control performance It was shown that by using theNNtechnique the control performance in both kinematic leveland dynamic level of the teleoperated robot was enhancedIn [81] an extreme learning machine (ELM) based controlstrategy was proposed for uncertain robot manipulators toidentify both the elasticity and geometry of an object ThisELM was applied to deal with the unknown nonlinearity ofthe robot manipulator to enhance the control performanceParticularly by utilizing ELM the proposed controller couldguarantee that the robot dynamics follow a reference modelsuch that the desired set point and the feedforward forcecould be updated to estimate the geometry and stiffness ofthe object As a result the reference model could be exactlymatched with a limited number of iterations

In [82] the NN controller was also employed to control awheel inverted pendulum which has been decomposed intotwo subsystems a fully actuated second-order planar movingsubsystem and a passive first-order pendulum subsystemThen the RBFNN was employed to compensate for theuncertain dynamics of the two subsystems by using itspowerful learning ability such that the enhanced controlperformance could be realized by using the NN learning In[83] a global adaptive neural control was proposed for a classof robot manipulators with finite-time convergence learn-ing performance This control scheme employed a smoothswitching mechanism combining with a nominal neuralnetwork controller and a robust controller to ensure globaluniform ultimately bounded stability The optimal weightswere obtained by the finite-time estimation algorithm suchthat after the learning process the learning weights could bereused next time for repeated tasks The global NN controlmechanism has been further extended to the control of dualarm robot manipulator in [84] where knowledge of bothrobot manipulator and the grasping object is unavailable inadvance By integrating prescribed functions into the designof controller the transient performance of the dual armrobot control was regularly guaranteed The NN was alsoemployed to deal with synchronization problem of multiplerobot manipulators in [85] where the reference trajectoriesare only available for part of the team members By usingthe NN approximation controller the robot has shown bettercontrol performance with enhanced transient performanceand enhanced robustness A RBFNN was constructed tocompensate for the nonlinear terms of a five-bar manipulatorbased on an error transformation function [86] The NNcontrol was also applied in the robot teleoperation control[87 88] Moreover a NN approximation technique wasemployed to deal with the unknown dynamics kinematicsand actuator properties in the manipulator tracking control[89]

42 NN Based Robot Control with Input NonlinearitiesAnother challenge of the robot manipulator is that theinput nonlinearities such as friction dead-zone and actuatorsaturation may inevitably exist in the robot systems Theseinput nonlinearities may lead to larger tracking errors anddegeneration of the control performance Therefore a num-ber of works have been proposed to handle the nonlinearitiesby utilizing the neural network design A neural adaptive

Complexity 7

controller was designed to deal with the effect of inputsaturation of the robot manipulator in [90] as follows

120591 = minus1199111 + 119863119878119863 (119885119863) 1205721 + 119862119878119862 (119885119862) 1205721+ 119866119878119866 (119885119866) + 119870119901 (1199112 + 120585) + 119870119903 sgn (1199112) (12)

where 1199111 is the robot position tracking error 1199112 is the velocitytracking error and 1205721 is an auxiliary controller 119863 119862 and119866 are the NN weights 119878119863(119885119863) 119878119862(119885119862) and 119878119866(119885119866) arethe NN regressor vectors and 119870119901 and 119870119903 are control gainsspecified by the designer 120585 is an auxiliary system designed toreduce the effect of the saturation with 120585 defined as follows

120585

= minus119870120585120585 minus

100381610038161003816100381610038161199111198792Δ12059110038161003816100381610038161003816 + (12) Δ120591119879Δ120591100381710038171003817100381712058510038171003817100381710038172 120585 + Δ120591 10038171003817100381710038171205851003817100381710038171003817 ge 1205830 10038171003817100381710038171205851003817100381710038171003817 lt 120583

(13)

where Δ120591 is the torque error caused by saturation and119870120585 is asmall positive value To update the NNweights adaptive lawsare designed as follows

119882119863 = Γ119863119896 (11987811986311989612057211198902119896 minus 120590119863119896119863119896)119882119862 = Γ119862119896 (11987811986211989612057211198902119896 minus 120590119862119896119862119896)119882119866 = Γ119866119896 (11987811986611989612057211198902119896 minus 120590119866119896119866119896)

(14)

where Γ119863119896 Γ119862119896 and Γ119866119896 are specified positive parameters and120590119863119896 120590119862119896 and 120590119866119896 are positive parametersIn [91] an adaptive neural network controller was con-

structed to approximate the input dead-zone and the uncer-tain dynamics of the robotic manipulator while the outputconstraint was also considered in the feedback control In[92] the NN was applied for the estimation of the unknownmodel parameters of a marine surface vessel and in [93] thefull-state constraint of an n-link robotic manipulator wasachieved by using the NN control The NN controller wasalso constructed for flexible roboticmanipulators to deal withthe vibration suppression based on a lumped spring-massmodel [94] while in [95] two RBFNNs were constructed forflexible robot manipulators to compensate for the unknowndynamics and the dead-zone effect respectively

The NN has also been used in many important industrialfields such as autonomous underwater vehicles (AUVs) andhypersonic flight vehicle (HFV) In [96] the NN has beenconstructed to deal with the attitude of AUVs in the presenceof input dead-zone and uncertain model parameters In [97]the adaptive neural control was employed to deal with under-water vehicle control in discrete-time domain encounteredwith the unknown input nonlinearities external disturbanceand model uncertainties Then the reinforcement learningwas applied to address these uncertainties by using a criticNN and an action NN The hypersonic flight vehicle controlwas investigated in [98] where the aerodynamic uncertaintiesand unknown disturbances were addressed by a disturbance

observer based NN In [99] a neural learning control wasembedded in the HFV controller to achieve the globalstability via a switching mechanism and a robust controller

43 NN Based Human-Robot Interaction Control Recentlythere is a predominant tendency to employ the robots inthe human-surrounded environment such as householdservices or industrial applications where humans and robotsmay interact with each other directly Therefore interactioncontrol has become a promising research field and has beenwidely studied In [100] a learning method was developedsuch that the dynamics of a robot arm could follow atarget impedance model with only knowledge of the roboticstructure (see Figure 6)

TheNNwas further employed in robot control in interac-tionwith an environment [101] where impedance control wasachieved with the completely unknown robotic dynamicsIn [102] a learning method was developed such that therobot was able to adjust the impedance parameters when itinteracted with unknown environments In order to learnoptimal impedance parameters in the robot manipulatorcontrol an adaptive dynamic programming (ADP) methodwas employed when the robot interacted with unknowntime-varying environments where NNs were used for bothcritic and actor networks [103] The ADP was also employedfor coordination of multirobots [104] in which possibledisagreement between different manipulators was handledand dynamics of both robots and themanipulated object werenot required to be known

In this work the controller consists of two parts a criticnetwork which was used to approximate the cost functionand an actual NN which was designed to control the robotThe critic NN is designed as follows [104]

Υ (119905) = 119862119878119862 (119885119862) (15)

where 119885119862 = 120585 120585 = [119879119900 119911119879 119909119879119900 ]119879 with 119909119900 being the positionof the object and 119911 being the tracking error 119862 is the NNweight and 119878119862 is the regressor vector

The critic NN is used to approximate a cost function119888(119905) = 120585119879119876120585 + 119906119879119877119906 where 119906 denotes the control inputand 119876 and 119877 are positive definite matrix Since the controlobjective is to minimize the control effort the adaptation lawis designed as

119882119888 = minus120590119888 120597119864119888120597119888 = 120590119888 (119888 minus 119879119888 119878119862) 119878119862 (16)

where 120590119888 is the learning rate and 119864119888 = (12)(119888 minus 119879119888 119878119862)2On the other hand the actual NN control is designed to

control the robot as

119906 = 119879119886 119878119886 (119885119886) minus 119890 minus 1198702119890V (17)

where 119879119886 119878119886(119885119886) could learn the dynamics of the robotwith 119886 being the NN weight and 119878119886 being the regressorvector 119890 and 119890V are the position and velocity tracking errorsrespectively and 1198702 is the control gain

8 Complexity

Positon control Robotic arm Inverse

kinematics

Forward kinematics

Optimal critic learning

EnvironmentImpedance

model

xd

xrqr

x q

f

minus

+

Figure 6 Block of the learning impedence control

Since the control objective is to guarantee the estimationof both robot dynamics and the cost function Υ(119905) theadaptive law is selected as follows

119882119879119886119894 = minus120590119886 (119879119886119894119878119886 + 119896119903Υ) 119878119886 (18)

where 120590119886 and 119896119903 are positive constantsOn the other hand as a fundamental element of the next-

generation robots the human-robot collaboration (HRC) hasbeen widely studied by roboticists and NN is employed inHRCwith its powerful learning ability In [105] theNNswereemployed to estimate the human partnerrsquos motion intentionin human-robot collaboration such that the robot was able toactively follow its human partner To adjust the robotrsquos role tolead or to follow according to the humanrsquos intention gametheory was employed for fundamental analysis of human-robot interaction and an adaptation law was developed in[106] Policy iteration combining with NN was adopted toprovide a rigorous solution to the problem of the systemequilibrium in human-robot interaction [107]

44 NN Based Robot Cognitive Control According to thepredictive processing theory [108] the human brain is alwaysactively anticipating the incoming sensorimotor informationThis process exists because the living beings exhibit latenciesdue to neural processing delays and a limited bandwidthin their sensorimotor processing To compensate for sucha delay in human brain neural feedback signals (includinglateral and top-down connections)modulate the neural activ-ities via inhibitory or excitatory connections by influencingthe neuronal population coding of the bottom-up sensory-driven signals in the perception-action system Similarlyin robotic systems it is claimed that such a delay and alimited bandwidth also can be compensated by the predictivefunctions learnt by recurrent neural models Such a learningprocess can be done via only visual processing [109] or in theloop of perception and action [110]

Based on the hierarchical sensorimotor integration the-ory which advocates that action and perception are inter-twined by sharing the same representational basis [111] therepresentation on different levels of sensory perception doesnot explicitly represent actions instead there is an encodingof the possible future percept which is learnt from priorsensorimotor knowledge

In the Bayesian once this perception and action linkshave been established after learning these perception-action

associations in this architecture allow the following opera-tions

First these associations allow predicting the perceptualoutcome of given actions by means of the forward models(eg Bayesian model) It can be written as

119875 (119864 | 119860 119868) prop 119875 (119860 | 119864) 119875 (119864 | 119868) (19)

where E estimates the upcoming perception evidence givenan executed action A and other prior information you havealready known in 119868 The term 119875(119864 | 119860 119868) suggests that aprelearntmodel representing the possibility of amotor actionA will be executed given that a (possible) resulting sensoryevidence 119864 is perceived (backward computation)

Second these associations allow selecting an appropri-ate movement given an intended perceptual representationFrom the backward computations introduced in the followingequation a predictive sensorimotor integration occurs

119875 (119860 | 119864 119866) prop 119875 (119864 | 119860) 119875 (119860 | 119866) (20)

where A indicates a particular action selected given the(intended) sensory information 119864 and a goal G Here weassume that onersquos action is only determined by the currentsensory input and the goal

In terms of its hierarchical organization it also allows thisoperation with bidirectional information pathways a lowlevel perception representation can be expressed on a higherlevel with a more complex receptive field and vice versa119890low hArr 119890high This can be realized by the bidirectional deeparchitectures such as [112] Conceptually these operationscan be achieved by extracting statistical regularity shown inFigure 7

Since both perception and action processes can be seenas temporal sequences from the mathematical perspectivethe recurrent networks are Turing-Complete and have alearning capacity to learn time sequences with arbitrarylength [113] if properly trained Furthermore such recurrentconnections can be placed in a hierarchical way in whichthe prediction functions on different layers attempt to predictthe nonlinear time-series in different time-scales [114] Fromthis point the recurrent neural network with parametric biasunits (RNNPB) [115] andmultiple time-scale recurrent neuralnetworks (MTRNN) [116] were applied to predict sequencesby understanding them in various temporal levels

The difference of the temporal levels controls the prop-erties of the different levels of the presentation in the deep

Complexity 9

SMA

M1

Brainstem

Spine

Action

Common coding

Common coding

MST

V5MT

V3

V1

Cognitiveprocesses

Visual perception(dorsal)

Figure 7 The process of cognitive control

recurrent network For instance in the MTRNN network[112] the learning of each neuron follows the updating rule ofclassical firing rate models in which the activity of a neuronis determined by the average firing rate of all the connectedneurons Additionally the neuronal activity is also decayingover time following an updating rule of leaky integratormodel Assuming the i-th MTRNN neuron has the numberof N connections the current membrane potential status ofa neuron can be defined by both the previous activation andthe current synaptic inputs

119906119894119905+1 = (1 minus 1120591119894)119906119894119905 + 1120591119894 [sum119908119894119895119909119895119905] (21)

where 119908119894119895 represents the synaptic weight from the j-thneuron to the i-th neuron 119909119895119905 is the activity of j-th neuronat t-th time-step and 120591 is the time-scale parameter whichdetermines the decay rate of this neuron a larger 120591 meanstheir activities change slowly over time compared with thosewith a smaller time-scale parameter 120591

In [117] the concepts of predictive coding were discussedin detail where the learning generation and recognition ofactions can be conducted by means of the principle of pre-diction error minimization By using the predictive codingthe RNNPB and MTRNN are capable for both generatingown actions and recognizing the same actions performedby others Recently the study on neurorobotics experimentshas shown that the dynamic predictive coding scheme canbe used to address fluctuations in temporal patterns whentraining a recurrent neural network (RNN) model [118]This predictive coding scheme enables organisms to predictperceptual outcomes based on current intentions of actionsto the external environment and to forecast perceptualsequences corresponding to given intention states [118]

Based on this architecture two-layer RNN models wereutilized to extract visual information [119] and to under-stand intentions [120] or emotion status [121] in socialrobotics three-layer RNNmodels were used to integrate and

understand multimodal information for a humanoid iCubrobot [112 122] Moreover the predictive coding frameworkhas been extended to variational Bayes predictive codingMTRNN which can arbitrate between deterministic modeland probabilistic model by setting a metaparameter [123]Such extension could provide significant improvement indealing with noisy fluctuated sensory inputs which robots areexpected to experience in more real world setting In [124]a MTRNN was employed to control a humanoid robot andexperimental results have shown that by using only partialtraining data the control model can achieve generalizationby learning in a lower feature perception level

The hierarchical structure of RNN exhibits a greatlearning capacity to store multimodal information whichis beneficial for the robotic systems to understand and topredict in a complex environment As the future models andapplications the state-of-the-art deep learning techniques orthemotor actions of robotic systems can be further integratedinto this predictive architecture

5 Conclusion

In summary great achievements for control design of nonlin-ear system by means of neural networks have been gained inthe last two decades Despite the impossibility in identifyingor listing all the related contributions in this short reviewefforts have been made to summarize the recent progressin the area of NN control and its particular applications inthe robot learning control the robot interaction control andthe robot recognition control In this paper we have shownthat significant progress of NN has been made in control ofthe nonlinear systems in solving the optimization problemin approximating the system dynamics in dealing with theinput nonlinearities in human-robot interaction and in thepattern recognition All these developments accompany notonly the development of techniques in control and advancedmanufactures but also theatrical progress in constructingand developing the neural networks Although huge efforts

10 Complexity

have been made to embed the NN in practical control sys-tems there is still a large gap between the theory and practiceTo improve the feasibility and usability the evolutionarycomputing theory has been proposed to train the NNs It canautomatically find a near-optimal NN architecture and allowa NN to adapt its learning rule to its environment Howeverthe complex and long training process of the evolutionaryalgorithms deters their practical applications More effortsneed to be made to evolve the NN architecture and NNlearning technique in the control design On the otherhand in human brain the neural activities are modulatedvia inhibitory or excitatory connections by influencing theneuronal population coding of the bottom-up sensory-drivensignals in the perception-action system In this sense howto integrate the sensor-motor information into the networkto make NNs more feasible to adapt to the environment andto resemble the capacity of the human brain deserves furtherinvestigations

In conclusion a brief review on neural networks for thecomplex nonlinear systems is provided with adaptive neuralcontrol NN based dynamic programming evolution com-puting and their practical applications in the robotic fieldsWe believe this area may promote increasing investigationsin both theories and applications And emerging topics likedeep learning [125ndash128] big data [129ndash131] and cloud com-puting may be incorporated into the neural network controlfor complex systems for example deep neural networkscould be used to process massive amounts of unsuperviseddata in complex scenarios neural networks can be helpfulin reducing the data dimensionality and the optimization ofNN training may be employed to enhance the learning andadaptation performance of robots

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

Thisworkwas partially supported by theNational Nature Sci-ence Foundation (NSFC) under Grant 61473120 GuangdongProvincial Natural Science Foundation 2014A030313266International Science and Technology Collaboration Grant2015A050502017 Science andTechnology Planning Project ofGuangzhou 201607010006 State Key Laboratory of Roboticsand System (HIT) Grant SKLRS-2017-KF-13 and the Funda-mental Research Funds for the Central Universities

References

[1] F Gruau D Whitley and L Pyeatt ldquoA comparison betweencellular encoding and direct encoding for genetic neural net-worksrdquo inProceedings of the 1st annual conference on genetic pro-gramming pp 81ndash89 1996

[2] H de Garis ldquoAn artificial brain ATRrsquos CAM-Brain Project aimsto buildevolve an artificial brain with a million neural netmodules inside a trillion cell Cellular Automata Machinerdquo NewGeneration Computing vol 12 no 2 pp 215ndash221 1994

[3] W S McCulloch and W Pitts ldquoA logical calculus of the ideasimmanent in nervous activityrdquoBulletin ofMathematical Biologyvol 5 no 4 pp 115ndash133 1943

[4] C Yang S S Ge C Xiang T Chai and T H Lee ldquoOutput feed-back NN control for two classes of discrete-time systems withunknown control directions in a unified approachrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 19 no 11pp 1873ndash1886 2008

[5] R Kumar R K Aggarwal and J D Sharma ldquoEnergy analysis ofa building using artificial neural network A reviewrdquo Energy andBuildings vol 65 pp 352ndash358 2013

[6] C Alippi C De Russis and V Piuri ldquoA neural-network basedcontrol solution to air-fuel ratio control for automotive fuel-injection systemsrdquo IEEE Transactions on Systems Man andCybernetics Part C Applications and Reviews vol 33 no 2 pp259ndash268 2003

[7] E M Azoff Neural network time series forecasting of financialmarkets John Wiley amp Sons Inc 1994

[8] I Fister P N Suganthan J Fister et al ldquoArtificial neural net-work regression as a local search heuristic for ensemble strate-gies in differential evolutionrdquo Nonlinear Dynamics vol 84 no2 pp 895ndash914 2016

[9] Y Zhang S Li and H Guo ldquoA type of biased consensus-based distributed neural network for path planningrdquoNonlinearDynamics vol 89 no 3 pp 1803ndash1815 2017

[10] X Shi Z Wang and L Han ldquoFinite-time stochastic synchro-nization of time-delay neural networks with noise disturbancerdquoNonlinear Dynamics vol 88 no 4 pp 2747ndash2755 2017

[11] P He and Y Li ldquoHinfin synchronization of coupled reaction-diffusion neural networks with mixed delaysrdquo Complexity vol21 no S2 pp 42ndash53 2016

[12] Z Tu J Cao A Alsaedi F E Alsaadi and T Hayat ldquoGlobalLagrange stability of complex-valued neural networks of neutraltype with time-varying delaysrdquo Complexity vol 21 no S2 pp438ndash450 2016

[13] C Wang S Guo and Y Xu ldquoFormation of autapse connectedto neuron and its biological functionrdquo Complexity vol 2017Article ID 5436737 9 pages 2017

[14] J D J Rubio I Elias D R Cruz and J Pacheco ldquoUniform stableradial basis function neural network for the prediction in twomechatronic processesrdquo Neurocomputing vol 227 pp 122ndash1302017

[15] J d Rubio ldquoUSNFIS Uniform stable neuro fuzzy inferencesystemrdquo Neurocomputing vol 262 pp 57ndash66 2017

[16] Q Liu J Yin V C M Leung J-H Zhai Z Cai and J LinldquoApplying a new localized generalization error model to designneural networks trained with extreme learning machinerdquo Neu-ral Computing and Applications pp 1ndash8 2014

[17] R J de Jesus ldquoInterpolation neural network model of a manu-factured wind turbinerdquoNeural Computing and Applications pp1ndash12 2016

[18] C Mu and D Wang ldquoNeural-network-based adaptive guaran-teed cost control of nonlinear dynamical systems with matcheduncertaintiesrdquo Neurocomputing vol 245 pp 46ndash54 2017

[19] Z Lin DMa J Meng and L Chen ldquoRelative ordering learningin spiking neural network for pattern recognitionrdquo Neuro-computing 2017

[20] J Yu J Sang and X Gao ldquoMachine learning and signal pro-cessing for big multimedia analysisrdquo Neurocomputing vol 257pp 1ndash4 2017

Complexity 11

[21] T Zhang S S Ge and C C Hang ldquoAdaptive neural networkcontrol for strict-feedback nonlinear systems using backstep-ping designrdquo Automatica vol 36 no 12 pp 1835ndash1846 2000

[22] S S Ge and T Zhang ldquoNeural-network control of nonaffinenonlinear system with zero dynamics by state and outputfeedbackrdquo IEEE Transactions on Neural Networks and LearningSystems vol 14 no 4 pp 900ndash918 2003

[23] S S Ge C Yang and T H Lee ldquoAdaptive predictive controlusing neural network for a class of pure-feedback systems in dis-crete timerdquo IEEE Transactions on Neural Networks and LearningSystems vol 19 no 9 pp 1599ndash1614 2008

[24] F W Lewis S Jagannathan and A Yesildirak Neural networkcontrol of robotmanipulators and non-linear systems CRCPress1998

[25] S Jagannathan and F L Lewis ldquoIdentification of nonlineardynamical systems using multilayered neural networksrdquo Auto-matica vol 32 no 12 pp 1707ndash1712 1996

[26] D Vrabie and F Lewis ldquoNeural network approach tocontinuous-time direct adaptive optimal control for partiallyunknown nonlinear systemsrdquo Neural Networks vol 22 no 3pp 237ndash246 2009

[27] C Yang S S Ge and T H Lee ldquoOutput feedback adaptive con-trol of a class of nonlinear discrete-time systems with unknowncontrol directionsrdquo Automatica vol 45 no 1 pp 270ndash2762009

[28] C Yang Z Li and J Li ldquoTrajectory planning and optimizedadaptive control for a class of wheeled inverted pendulumvehicle modelsrdquo IEEE Transactions on Cybernetics vol 43 no 1pp 24ndash36 2013

[29] Y Jiang C Yang and H Ma ldquoA review of fuzzy logic andneural network based intelligent control design for discrete-time systemsrdquo Discrete Dynamics in Nature and Society ArticleID 7217364 Art ID 7217364 11 pages 2016

[30] Y Jiang C Yang S-L Dai and B Ren ldquoDeterministic learningenhanced neutral network control of unmanned helicopterrdquoInternational Journal of Advanced Robotic Systems vol 13 no6 pp 1ndash12 2016

[31] Y Jiang Z Liu C Chen and Y Zhang ldquoAdaptive robust fuzzycontrol for dual arm robot with unknown input deadzonenonlinearityrdquo Nonlinear Dynamics vol 81 no 3 pp 1301ndash13142015

[32] httpsstemcellthailandorgneurons-definition-function-neurotransmitters

[33] MDefoort T Floquet A Kokosy andW Perruquetti ldquoSliding-mode formation control for cooperative autonomous mobilerobotsrdquo IEEE Transactions on Industrial Electronics vol 55 no11 pp 3944ndash3953 2008

[34] X Liu C Yang Z Chen MWang and C Su ldquoNeuro-adaptiveobserver based control of flexible joint robotrdquoNeurocomputing2017

[35] F Hamerlain T Floquet and W Perruquetti ldquoExperimentaltests of a slidingmode controller for trajectory tracking of a car-like mobile robotrdquo Robotica vol 32 no 1 pp 63ndash76 2014

[36] R J de Jesus ldquoDiscrete time control based in neural networksfor pendulumsrdquo Applied Soft Computing 2017

[37] Y Pan M J Er T Sun B Xu and H Yu ldquoAdaptive fuzzy PDcontrol with stable Hinfin tracking guaranteerdquo Neurocomputingvol 237 pp 71ndash78 2017

[38] R J de Jesus ldquoAdaptive least square control in discrete time ofrobotic armsrdquo Soft Computing vol 19 no 12 pp 3665ndash36762015

[39] S Commuri S Jagannathan and F L Lewis ldquoCMAC neuralnetwork control of robot manipulatorsrdquo Journal of RoboticSystems vol 14 no 6 pp 465ndash482 1997

[40] J S Albus ldquoTheoretical and experimental aspects of a cerebellarmodelrdquoDevelopmental Disabilities Research Reviews vol 17 pp93ndash101 1972

[41] B Yang R Bao and H Han ldquoRobust hybrid control based onPD and novel CMAC with improved architecture and learningscheme for electric load simulatorrdquo IEEE Transactions onIndustrial Electronics vol 61 no 10 pp 5271ndash5279 2014

[42] S Jagannathan and F L Lewis ldquoMultilayer discrete-timeneural-net controller with guaranteed performancerdquo IEEETransactions on Neural Networks and Learning Systems vol 7no 1 pp 107ndash130 1996

[43] S S Ge and J Wang ldquoRobust adaptive neural control for a classof perturbed strict feedback nonlinear systemsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no6 pp 1409ndash1419 2002

[44] Y H Kim F L Lewis and C T Abdallah ldquoA dynamic recurrentneural-network-based adaptive observer for a class of nonlinearsystemsrdquo Automatica vol 33 no 8 pp 1539ndash1543 1997

[45] J-Q Huang and F L Lewis ldquoNeural-network predictive controlfor nonlinear dynamic systems with time-delayrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 14 no 2pp 377ndash389 2003

[46] S S Ge F Hong and T H Lee ldquoAdaptive neural control ofnonlinear time-delay systems with unknown virtual controlcoefficientsrdquo IEEE Transactions on Systems Man and Cybernet-ics Part B Cybernetics vol 34 no 1 pp 499ndash516 2004

[47] J Na X M Ren and H Huang ldquoTime-delay positive feedbackcontrol for nonlinear time-delay systems with neural networkcompensationrdquo Zidonghua XuebaoActa Automatica Sinica vol34 no 9 pp 1196ndash1202 2008

[48] J Na X Ren C Shang and Y Guo ldquoAdaptive neural networkpredictive control for nonlinear pure feedback systems withinput delayrdquo Journal of Process Control vol 22 no 1 pp 194ndash206 2012

[49] R R Selmic and F L Lewis ldquoNeural-network approximationof piecewise continuous functions application to friction com-pensationrdquo IEEE Transactions on Neural Networks and LearningSystems vol 13 no 3 pp 745ndash751 2002

[50] H Zhang and F L Lewis ldquoAdaptive cooperative trackingcontrol of higher-order nonlinear systems with unknowndynamicsrdquo Automatica vol 48 no 7 pp 1432ndash1439 2012

[51] J Na X Ren and D Zheng ldquoAdaptive control for nonlinearpure-feedback systems with high-order sliding mode observerrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 24 no 3 pp 370ndash382 2013

[52] J Na Q Chen X Ren and Y Guo ldquoAdaptive prescribedperformancemotion control of servomechanisms with frictioncompensationrdquo IEEE Transactions on Industrial Electronics vol61 no 1 pp 486ndash494 2014

[53] J Na X Ren G Herrmann and Z Qiao ldquoAdaptive neuraldynamic surface control for servo systems with unknown dead-zonerdquoControl Engineering Practice vol 19 no 11 pp 1328ndash13432011

[54] G Li J Na D P Stoten and X Ren ldquoAdaptive neural networkfeedforward control for dynamically substructured systemsrdquoIEEE Transactions on Control Systems Technology vol 22 no3 pp 944ndash954 2014

4 Complexity

3 Theoretical Developments

31 Adaptive Neural Control During the past two decadesvarious neural networks have been incorporated into adap-tive control for nonlinear systems with unknown dynamicsIn [42] a multiplayer discrete-time neural network con-troller was constructed for a class of multi-input multioutput(MIMO) dynamical systems where NNweights were trainedusing an improved online tuning algorithm An adaptiveNN output feedback control was proposed to control twoclasses of discrete-time systems in the presence of unknowncontrol directions [4] A robust adaptive neural controllerwas developed for a class of strict-feedback systems in[43] where a Nussbaum gain technique was employed todeal with unknown virtual control coefficients A dynamicrecurrent NN was employed for construction of an adaptiveobserver with online turned weights parameters in [44] andto deal with the time-delay of a class of nonlinear dynamicalsystems in [45] The time-delay of strict-feedback nonlinearsystems was also addressed by using NN control with properdesigned Lyapunov-Krasovskii functions in [46] For a classof unknown nonlinear affine time-delay systems an adaptivecontrol schemewas proposed by constructing two high-orderNNs for identifying system uncertainties [47] This idea hasbeen further extended to affine nonlinear systems with inputtime-delay in [48]

It should be noticed that piecewise continuous func-tions such as frictions backlash and dead-zone are widelyexisted in industrial plants Other than continuous nonlinearfunction the approximation of these piecewise functions ismore challenging since the NNrsquos universal approximationonly holds for continues functions To approximate thesepiecewise continuous functions a novel NN structure wasdesigned by involving a standard activation function and ajump approximation basis function [49] In [47] a CMACNN was employed for the closed-loop control of nonlineardynamical systems with rigorous stability analysis and in[50] a robust adaptive neural network control scheme wasdeveloped for cooperative tracking control of higher-ordernonlinear systems

The adaptive NN control scheme was also proposed forpure-feedback systems In [51] a high-order sliding modeobserverwas proposed to estimate the unknown system stateswhile two NNs were constructed to deal with approximationerrors and the unknown nonlinearities respectively In com-parison to the conventional control design for pure-feedbacksystems the state-feedback control was achieved withoutusing the backstepping technique In [52] a neural controlframework was proposed for nonlinear servo mechanism toguarantee both the steady-state and transient tracking per-formance In this work a prescribed performance functionwas employed in an output error transformation such thatthe tracking performance can be guaranteed by the regulationcontrol of the outputs In [53] an adaptive neural control wasalso designed for a class of nonlinear systems in the presenceof time-delays and input dead-zone and high-order neuralnetworks were employed to deal the unknown uncertaintiesIn this work a salient feature lies in the fact that only thenorm of the NNsrsquo weights (a scalar) needs to be online

updated such that the computational efficiency in the onlineimplementation could be significantly improved In [54] theauthors developed a neural network based feedforward con-trol to compensate for the nonlinearities and uncertaintiesof a dynamically substructured system consisting of bothnumerical and physical substructures where an adaptive lawwith a new leakage term ofNNweights error informationwasdeveloped to achieve improved convergence An experimenton a quasi-motorcycle testing rig validated the efficacy ofthis control strategy In [55] a neural dynamic control wasincorporated into the strict-feedback control of a class ofunknown nonlinear systems by using the dynamic surfacecontrol technique For a class of uncertain nonlinear systemswith unknown hysteresis NN was used for compensationof the nonlinearities [56] In [57] to deal with unknownnonsymmetrical input saturations of unknown nonaffinesystems NNs were used in the stateoutput feedback controlbased on the mean value theorem and the implicit functionTo avoid using the backstepping synthesis a dynamic surfacecontrol scheme was designed by combining the NN with anonlinear disturbance observer [58]

32 NN Based Adaptive Dynamic Programming In additionto adaptive control neural networks have also been adoptedto solve the optimization problem for nonlinear systemsIn convention optimal control the dynamic programmingmethod was widely used It aims to minimize a predefinedcost function such that a sequence of optimal control inputscould be derived However the cost function is usuallydifficult to online calculate due to the computation com-plexity in obtaining the solution of the Hamilton-Jacobi-Bellman (HJB) equationTherefore an adaptiveapproximatedynamic programming (ADP) technique was developed in[59] where a NN was trained to estimate the cost functionand then to derive solutions for the ADP Generally theADP has several different synonyms including approximatedynamic programming heuristic dynamic programming(HDP) critic network and reinforcement learning (RL) [60ndash62] Figure 4 shows the basic framework of the HDP with acritic-actor structure In [63] a discrete-time HJB equationwas solved using an NN based HDP algorithm to derive theoptimal control of nonlinear discrete-time systems In [64]three neural networks were constructed for an iterative ADPsuch that optimal feedback control of a discrete-time affinenonlinear system could be realized In [65] a globalized dualheuristic programming was presented to address the optimalcontrol of discrete-time systems In each iteration threeneural networks were used to learn the cost function andthe unknown nonlinear systems In [66] a reference networkcombining with an action network and a critic network wasintroduced in the ADP architecture to derive an internalgoal representation such that the learning and optimizationprocess could be facilitated The reference network has alsobeen introduced in the online action-dependent heuristicdynamic programming by employing a dual critic networkframework A policy iteration algorithm was introduced forinfinite horizon optimal control of nonlinear systems usingADP in [67] In [68] a reinforcement learning method wasintroduced for the stabilizing control of uncertain nonlinear

Complexity 5

Action network

Modelnetwork

Critic network

Critic network

x(k)u(k)

x(k + 1) U(k)

J(x(k))

J(x(k + 1))

minus

Figure 4 An overview of the HDP structure

systems in the presence of input constraints By using this RL-based controller a constrained optimal control problem wassolved with construction of only one critic neural network In[69] an ADP technique for online control and learning of ageneralized multiple-input-multiple-output (MIMO) systemwas investigated In [70] an adaptive NN based ADP controlscheme was presented for a class of nonlinear systems withunknown dynamics The optimal control law was calculatedby using a dual neural network scheme with a critic NN andan identifier NN Particularly parameters estimation errorwas used to online identify the learning weights to achievethe finite-time convergence Optimal tracking control for aclass of nonlinear systems was investigated in [71] where anew ldquoidentifier-criticrdquo based ADP framework was proposed

33 Evolutionary Computing In addition to the capacity ofapproximation and optimization of the NN there has beenalso a great interest in using the evolutionary approachesto train the neural networks With the evolvement of NNarchitectures learning rules connection weights and inputfeatures an evolutionary artificial neural network (EANN)was designed to provide superior performance in comparisonto conventional training approaches [72] A literature reviewof the EANNwas given in [73] where the evolution strategiessuch as feedforward artificial NN and genetic algorithms(GA) have been introduced for the EANNs In [72] severalEANN frameworks were introduced by embedding the evo-lution algorithms (EA) to evolve the NN structure In [74]an EPNet evolution system was proposed for evolving thefeedforward NN based on Fogelrsquos evolutionary programming(EP) method which could improve the NNrsquos connectionweights and architectures at the same time as well as decreasenoise in the fitness evaluation Good generalization abilityof the evolved NN has been constructed and verified inthe experiments In [75] a GA based technique has beenemployed to train the NNs in direct neural control systemssuch that the NN architectures could be optimized Adeficiency of the EANN is that the optimization processwould often result in a low training speed To overcomethis problem and facilitate adaptation processes a hybridmultiobjective evolutionary method was developed in [76]where the singular-value-decomposition (SVD) techniquewas employed to choose the necessary neurons number in thetraining of a feedforwardNNThe evolutionary approachwas

applied to identify a grey-box model with a multiobjectiveoptimization between the clearly known practical systemsand approximated nonlinear systems [77] Applications ofevolutionary algorithms for robotic navigation have beenintroduced and investigated in [78] A survey of machinelearning technique was reported in [79] where several meth-ods to improve the evolutionary computation were reviewed

The evolution algorithms have been employed in manyaspects for evolvements of NNs such as to train the NN con-nection weights or to obtain near-optimal NN architecturesas well as adapting learning rules of NNs to their environ-ment In a word the evolution algorithms provide NNs withthe ability of learning to learn and also to build the relation-ship between evolution and learning such that the EANNcould perform favorable ability to adapt to changes of thedynamic environment

4 Applications in Robots

41 NNBased RoboticManipulator Control Generally speak-ing the control methods for robot manipulators can beroughly divided into two groups model-free control andmodel based control For the model-free control approacheslike proportional-integral-derivative (PID) control satisfac-tory control performancemay not be guaranteed In contrastthe model based control approaches exhibit better controlbehavior but heavily depend on the validity of the robotmodel In practice however a perfect robotic dynamicmodel is always not available due to the complex mech-anisms and uncertainties Additionally the payload maybe varied according to different tasks which makes theaccurate dynamics model hard to be obtained in advance Tosolve such problems the NN approximation-based controlmethods have been used extensively in applications of robotmanipulator control A basic structure of the adaptive neuralnetwork control for robot manipulator is shown in Figure 5Consider a dynamic model of a robot manipulator given asfollows [80]

119872(119902) 119902 + 119862 ( 119902 119902) 119902 + 119866 (119902) = 120591 (9)

where119872(119902)119862( 119902 119902) and119866(119902) are the inertialmatrix Coriolismatrix and gravity vector respectively Then NN controldesign could be given as follows

120591119889 = minus11987011198901 minus 11987021198902 + 119878 (119885) (10)

6 Complexity

PD controller Robotic arm

Closed loop neural network controller

Neural network controller

Neuralinputs

qd qd e e q q q1

d

Figure 5 A basic structure of the adaptive NN robot control

where 1198901 is the tracking error 1198902 is the velocity tracking error119878(119885) is the NN controller with being the weights matrixand 119878(119885) being the NN regressor vector and 1198701 and 1198702 arecontrol gains specified by the designer

From (10) we can see that the robot controller consists ofa PD-like controller and aNN controller In traditionalmodelbased controllers the dynamic model of the robot could beregarded as a feedforward to address the effect caused bythe robot motion In practice however 119872(119902) 119862(119902 119902) and119866(119902) may not be known Therefore the NNs are used toapproximate the unknown dynamics 119891 = 119872(119902) 119902 +119862( 119902 119902) 119902 +119866(119902) and to improve the performance of the system via theonline estimation To adapt theNNweights adaptive laws aredesigned as follows

119882 = Γ (119878119890 minus 120590) (11)

where Γ and 120590 are specified positive parametersThe last termof right-hand side of (11) is the sigma modification whichis used to enhance the convergence and robustness of theparameters adaptation

In [80] a NN based share control method was devel-oped to control a teleoperated robot with environmentaluncertainties In this work the RBFNN was constructed tocompensate for the unknown dynamics of the teleoperatedrobot Particularly a shared control strategy was developedinto the controller to achieve the automatic obstacle avoid-ance combining with the information of visual camera andthe robot body such that the obstacle could be successfullyavoided and the operator could focus more on the operatedtask rather than the environment to guarantee the stabilityand manipulation In addition error transformations wereintegrated into the adaptiveNN control to guarantee the tran-sient control performance It was shown that by using theNNtechnique the control performance in both kinematic leveland dynamic level of the teleoperated robot was enhancedIn [81] an extreme learning machine (ELM) based controlstrategy was proposed for uncertain robot manipulators toidentify both the elasticity and geometry of an object ThisELM was applied to deal with the unknown nonlinearity ofthe robot manipulator to enhance the control performanceParticularly by utilizing ELM the proposed controller couldguarantee that the robot dynamics follow a reference modelsuch that the desired set point and the feedforward forcecould be updated to estimate the geometry and stiffness ofthe object As a result the reference model could be exactlymatched with a limited number of iterations

In [82] the NN controller was also employed to control awheel inverted pendulum which has been decomposed intotwo subsystems a fully actuated second-order planar movingsubsystem and a passive first-order pendulum subsystemThen the RBFNN was employed to compensate for theuncertain dynamics of the two subsystems by using itspowerful learning ability such that the enhanced controlperformance could be realized by using the NN learning In[83] a global adaptive neural control was proposed for a classof robot manipulators with finite-time convergence learn-ing performance This control scheme employed a smoothswitching mechanism combining with a nominal neuralnetwork controller and a robust controller to ensure globaluniform ultimately bounded stability The optimal weightswere obtained by the finite-time estimation algorithm suchthat after the learning process the learning weights could bereused next time for repeated tasks The global NN controlmechanism has been further extended to the control of dualarm robot manipulator in [84] where knowledge of bothrobot manipulator and the grasping object is unavailable inadvance By integrating prescribed functions into the designof controller the transient performance of the dual armrobot control was regularly guaranteed The NN was alsoemployed to deal with synchronization problem of multiplerobot manipulators in [85] where the reference trajectoriesare only available for part of the team members By usingthe NN approximation controller the robot has shown bettercontrol performance with enhanced transient performanceand enhanced robustness A RBFNN was constructed tocompensate for the nonlinear terms of a five-bar manipulatorbased on an error transformation function [86] The NNcontrol was also applied in the robot teleoperation control[87 88] Moreover a NN approximation technique wasemployed to deal with the unknown dynamics kinematicsand actuator properties in the manipulator tracking control[89]

42 NN Based Robot Control with Input NonlinearitiesAnother challenge of the robot manipulator is that theinput nonlinearities such as friction dead-zone and actuatorsaturation may inevitably exist in the robot systems Theseinput nonlinearities may lead to larger tracking errors anddegeneration of the control performance Therefore a num-ber of works have been proposed to handle the nonlinearitiesby utilizing the neural network design A neural adaptive

Complexity 7

controller was designed to deal with the effect of inputsaturation of the robot manipulator in [90] as follows

120591 = minus1199111 + 119863119878119863 (119885119863) 1205721 + 119862119878119862 (119885119862) 1205721+ 119866119878119866 (119885119866) + 119870119901 (1199112 + 120585) + 119870119903 sgn (1199112) (12)

where 1199111 is the robot position tracking error 1199112 is the velocitytracking error and 1205721 is an auxiliary controller 119863 119862 and119866 are the NN weights 119878119863(119885119863) 119878119862(119885119862) and 119878119866(119885119866) arethe NN regressor vectors and 119870119901 and 119870119903 are control gainsspecified by the designer 120585 is an auxiliary system designed toreduce the effect of the saturation with 120585 defined as follows

120585

= minus119870120585120585 minus

100381610038161003816100381610038161199111198792Δ12059110038161003816100381610038161003816 + (12) Δ120591119879Δ120591100381710038171003817100381712058510038171003817100381710038172 120585 + Δ120591 10038171003817100381710038171205851003817100381710038171003817 ge 1205830 10038171003817100381710038171205851003817100381710038171003817 lt 120583

(13)

where Δ120591 is the torque error caused by saturation and119870120585 is asmall positive value To update the NNweights adaptive lawsare designed as follows

119882119863 = Γ119863119896 (11987811986311989612057211198902119896 minus 120590119863119896119863119896)119882119862 = Γ119862119896 (11987811986211989612057211198902119896 minus 120590119862119896119862119896)119882119866 = Γ119866119896 (11987811986611989612057211198902119896 minus 120590119866119896119866119896)

(14)

where Γ119863119896 Γ119862119896 and Γ119866119896 are specified positive parameters and120590119863119896 120590119862119896 and 120590119866119896 are positive parametersIn [91] an adaptive neural network controller was con-

structed to approximate the input dead-zone and the uncer-tain dynamics of the robotic manipulator while the outputconstraint was also considered in the feedback control In[92] the NN was applied for the estimation of the unknownmodel parameters of a marine surface vessel and in [93] thefull-state constraint of an n-link robotic manipulator wasachieved by using the NN control The NN controller wasalso constructed for flexible roboticmanipulators to deal withthe vibration suppression based on a lumped spring-massmodel [94] while in [95] two RBFNNs were constructed forflexible robot manipulators to compensate for the unknowndynamics and the dead-zone effect respectively

The NN has also been used in many important industrialfields such as autonomous underwater vehicles (AUVs) andhypersonic flight vehicle (HFV) In [96] the NN has beenconstructed to deal with the attitude of AUVs in the presenceof input dead-zone and uncertain model parameters In [97]the adaptive neural control was employed to deal with under-water vehicle control in discrete-time domain encounteredwith the unknown input nonlinearities external disturbanceand model uncertainties Then the reinforcement learningwas applied to address these uncertainties by using a criticNN and an action NN The hypersonic flight vehicle controlwas investigated in [98] where the aerodynamic uncertaintiesand unknown disturbances were addressed by a disturbance

observer based NN In [99] a neural learning control wasembedded in the HFV controller to achieve the globalstability via a switching mechanism and a robust controller

43 NN Based Human-Robot Interaction Control Recentlythere is a predominant tendency to employ the robots inthe human-surrounded environment such as householdservices or industrial applications where humans and robotsmay interact with each other directly Therefore interactioncontrol has become a promising research field and has beenwidely studied In [100] a learning method was developedsuch that the dynamics of a robot arm could follow atarget impedance model with only knowledge of the roboticstructure (see Figure 6)

TheNNwas further employed in robot control in interac-tionwith an environment [101] where impedance control wasachieved with the completely unknown robotic dynamicsIn [102] a learning method was developed such that therobot was able to adjust the impedance parameters when itinteracted with unknown environments In order to learnoptimal impedance parameters in the robot manipulatorcontrol an adaptive dynamic programming (ADP) methodwas employed when the robot interacted with unknowntime-varying environments where NNs were used for bothcritic and actor networks [103] The ADP was also employedfor coordination of multirobots [104] in which possibledisagreement between different manipulators was handledand dynamics of both robots and themanipulated object werenot required to be known

In this work the controller consists of two parts a criticnetwork which was used to approximate the cost functionand an actual NN which was designed to control the robotThe critic NN is designed as follows [104]

Υ (119905) = 119862119878119862 (119885119862) (15)

where 119885119862 = 120585 120585 = [119879119900 119911119879 119909119879119900 ]119879 with 119909119900 being the positionof the object and 119911 being the tracking error 119862 is the NNweight and 119878119862 is the regressor vector

The critic NN is used to approximate a cost function119888(119905) = 120585119879119876120585 + 119906119879119877119906 where 119906 denotes the control inputand 119876 and 119877 are positive definite matrix Since the controlobjective is to minimize the control effort the adaptation lawis designed as

119882119888 = minus120590119888 120597119864119888120597119888 = 120590119888 (119888 minus 119879119888 119878119862) 119878119862 (16)

where 120590119888 is the learning rate and 119864119888 = (12)(119888 minus 119879119888 119878119862)2On the other hand the actual NN control is designed to

control the robot as

119906 = 119879119886 119878119886 (119885119886) minus 119890 minus 1198702119890V (17)

where 119879119886 119878119886(119885119886) could learn the dynamics of the robotwith 119886 being the NN weight and 119878119886 being the regressorvector 119890 and 119890V are the position and velocity tracking errorsrespectively and 1198702 is the control gain

8 Complexity

Positon control Robotic arm Inverse

kinematics

Forward kinematics

Optimal critic learning

EnvironmentImpedance

model

xd

xrqr

x q

f

minus

+

Figure 6 Block of the learning impedence control

Since the control objective is to guarantee the estimationof both robot dynamics and the cost function Υ(119905) theadaptive law is selected as follows

119882119879119886119894 = minus120590119886 (119879119886119894119878119886 + 119896119903Υ) 119878119886 (18)

where 120590119886 and 119896119903 are positive constantsOn the other hand as a fundamental element of the next-

generation robots the human-robot collaboration (HRC) hasbeen widely studied by roboticists and NN is employed inHRCwith its powerful learning ability In [105] theNNswereemployed to estimate the human partnerrsquos motion intentionin human-robot collaboration such that the robot was able toactively follow its human partner To adjust the robotrsquos role tolead or to follow according to the humanrsquos intention gametheory was employed for fundamental analysis of human-robot interaction and an adaptation law was developed in[106] Policy iteration combining with NN was adopted toprovide a rigorous solution to the problem of the systemequilibrium in human-robot interaction [107]

44 NN Based Robot Cognitive Control According to thepredictive processing theory [108] the human brain is alwaysactively anticipating the incoming sensorimotor informationThis process exists because the living beings exhibit latenciesdue to neural processing delays and a limited bandwidthin their sensorimotor processing To compensate for sucha delay in human brain neural feedback signals (includinglateral and top-down connections)modulate the neural activ-ities via inhibitory or excitatory connections by influencingthe neuronal population coding of the bottom-up sensory-driven signals in the perception-action system Similarlyin robotic systems it is claimed that such a delay and alimited bandwidth also can be compensated by the predictivefunctions learnt by recurrent neural models Such a learningprocess can be done via only visual processing [109] or in theloop of perception and action [110]

Based on the hierarchical sensorimotor integration the-ory which advocates that action and perception are inter-twined by sharing the same representational basis [111] therepresentation on different levels of sensory perception doesnot explicitly represent actions instead there is an encodingof the possible future percept which is learnt from priorsensorimotor knowledge

In the Bayesian once this perception and action linkshave been established after learning these perception-action

associations in this architecture allow the following opera-tions

First these associations allow predicting the perceptualoutcome of given actions by means of the forward models(eg Bayesian model) It can be written as

119875 (119864 | 119860 119868) prop 119875 (119860 | 119864) 119875 (119864 | 119868) (19)

where E estimates the upcoming perception evidence givenan executed action A and other prior information you havealready known in 119868 The term 119875(119864 | 119860 119868) suggests that aprelearntmodel representing the possibility of amotor actionA will be executed given that a (possible) resulting sensoryevidence 119864 is perceived (backward computation)

Second these associations allow selecting an appropri-ate movement given an intended perceptual representationFrom the backward computations introduced in the followingequation a predictive sensorimotor integration occurs

119875 (119860 | 119864 119866) prop 119875 (119864 | 119860) 119875 (119860 | 119866) (20)

where A indicates a particular action selected given the(intended) sensory information 119864 and a goal G Here weassume that onersquos action is only determined by the currentsensory input and the goal

In terms of its hierarchical organization it also allows thisoperation with bidirectional information pathways a lowlevel perception representation can be expressed on a higherlevel with a more complex receptive field and vice versa119890low hArr 119890high This can be realized by the bidirectional deeparchitectures such as [112] Conceptually these operationscan be achieved by extracting statistical regularity shown inFigure 7

Since both perception and action processes can be seenas temporal sequences from the mathematical perspectivethe recurrent networks are Turing-Complete and have alearning capacity to learn time sequences with arbitrarylength [113] if properly trained Furthermore such recurrentconnections can be placed in a hierarchical way in whichthe prediction functions on different layers attempt to predictthe nonlinear time-series in different time-scales [114] Fromthis point the recurrent neural network with parametric biasunits (RNNPB) [115] andmultiple time-scale recurrent neuralnetworks (MTRNN) [116] were applied to predict sequencesby understanding them in various temporal levels

The difference of the temporal levels controls the prop-erties of the different levels of the presentation in the deep

Complexity 9

SMA

M1

Brainstem

Spine

Action

Common coding

Common coding

MST

V5MT

V3

V1

Cognitiveprocesses

Visual perception(dorsal)

Figure 7 The process of cognitive control

recurrent network For instance in the MTRNN network[112] the learning of each neuron follows the updating rule ofclassical firing rate models in which the activity of a neuronis determined by the average firing rate of all the connectedneurons Additionally the neuronal activity is also decayingover time following an updating rule of leaky integratormodel Assuming the i-th MTRNN neuron has the numberof N connections the current membrane potential status ofa neuron can be defined by both the previous activation andthe current synaptic inputs

119906119894119905+1 = (1 minus 1120591119894)119906119894119905 + 1120591119894 [sum119908119894119895119909119895119905] (21)

where 119908119894119895 represents the synaptic weight from the j-thneuron to the i-th neuron 119909119895119905 is the activity of j-th neuronat t-th time-step and 120591 is the time-scale parameter whichdetermines the decay rate of this neuron a larger 120591 meanstheir activities change slowly over time compared with thosewith a smaller time-scale parameter 120591

In [117] the concepts of predictive coding were discussedin detail where the learning generation and recognition ofactions can be conducted by means of the principle of pre-diction error minimization By using the predictive codingthe RNNPB and MTRNN are capable for both generatingown actions and recognizing the same actions performedby others Recently the study on neurorobotics experimentshas shown that the dynamic predictive coding scheme canbe used to address fluctuations in temporal patterns whentraining a recurrent neural network (RNN) model [118]This predictive coding scheme enables organisms to predictperceptual outcomes based on current intentions of actionsto the external environment and to forecast perceptualsequences corresponding to given intention states [118]

Based on this architecture two-layer RNN models wereutilized to extract visual information [119] and to under-stand intentions [120] or emotion status [121] in socialrobotics three-layer RNNmodels were used to integrate and

understand multimodal information for a humanoid iCubrobot [112 122] Moreover the predictive coding frameworkhas been extended to variational Bayes predictive codingMTRNN which can arbitrate between deterministic modeland probabilistic model by setting a metaparameter [123]Such extension could provide significant improvement indealing with noisy fluctuated sensory inputs which robots areexpected to experience in more real world setting In [124]a MTRNN was employed to control a humanoid robot andexperimental results have shown that by using only partialtraining data the control model can achieve generalizationby learning in a lower feature perception level

The hierarchical structure of RNN exhibits a greatlearning capacity to store multimodal information whichis beneficial for the robotic systems to understand and topredict in a complex environment As the future models andapplications the state-of-the-art deep learning techniques orthemotor actions of robotic systems can be further integratedinto this predictive architecture

5 Conclusion

In summary great achievements for control design of nonlin-ear system by means of neural networks have been gained inthe last two decades Despite the impossibility in identifyingor listing all the related contributions in this short reviewefforts have been made to summarize the recent progressin the area of NN control and its particular applications inthe robot learning control the robot interaction control andthe robot recognition control In this paper we have shownthat significant progress of NN has been made in control ofthe nonlinear systems in solving the optimization problemin approximating the system dynamics in dealing with theinput nonlinearities in human-robot interaction and in thepattern recognition All these developments accompany notonly the development of techniques in control and advancedmanufactures but also theatrical progress in constructingand developing the neural networks Although huge efforts

10 Complexity

have been made to embed the NN in practical control sys-tems there is still a large gap between the theory and practiceTo improve the feasibility and usability the evolutionarycomputing theory has been proposed to train the NNs It canautomatically find a near-optimal NN architecture and allowa NN to adapt its learning rule to its environment Howeverthe complex and long training process of the evolutionaryalgorithms deters their practical applications More effortsneed to be made to evolve the NN architecture and NNlearning technique in the control design On the otherhand in human brain the neural activities are modulatedvia inhibitory or excitatory connections by influencing theneuronal population coding of the bottom-up sensory-drivensignals in the perception-action system In this sense howto integrate the sensor-motor information into the networkto make NNs more feasible to adapt to the environment andto resemble the capacity of the human brain deserves furtherinvestigations

In conclusion a brief review on neural networks for thecomplex nonlinear systems is provided with adaptive neuralcontrol NN based dynamic programming evolution com-puting and their practical applications in the robotic fieldsWe believe this area may promote increasing investigationsin both theories and applications And emerging topics likedeep learning [125ndash128] big data [129ndash131] and cloud com-puting may be incorporated into the neural network controlfor complex systems for example deep neural networkscould be used to process massive amounts of unsuperviseddata in complex scenarios neural networks can be helpfulin reducing the data dimensionality and the optimization ofNN training may be employed to enhance the learning andadaptation performance of robots

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

Thisworkwas partially supported by theNational Nature Sci-ence Foundation (NSFC) under Grant 61473120 GuangdongProvincial Natural Science Foundation 2014A030313266International Science and Technology Collaboration Grant2015A050502017 Science andTechnology Planning Project ofGuangzhou 201607010006 State Key Laboratory of Roboticsand System (HIT) Grant SKLRS-2017-KF-13 and the Funda-mental Research Funds for the Central Universities

References

[1] F Gruau D Whitley and L Pyeatt ldquoA comparison betweencellular encoding and direct encoding for genetic neural net-worksrdquo inProceedings of the 1st annual conference on genetic pro-gramming pp 81ndash89 1996

[2] H de Garis ldquoAn artificial brain ATRrsquos CAM-Brain Project aimsto buildevolve an artificial brain with a million neural netmodules inside a trillion cell Cellular Automata Machinerdquo NewGeneration Computing vol 12 no 2 pp 215ndash221 1994

[3] W S McCulloch and W Pitts ldquoA logical calculus of the ideasimmanent in nervous activityrdquoBulletin ofMathematical Biologyvol 5 no 4 pp 115ndash133 1943

[4] C Yang S S Ge C Xiang T Chai and T H Lee ldquoOutput feed-back NN control for two classes of discrete-time systems withunknown control directions in a unified approachrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 19 no 11pp 1873ndash1886 2008

[5] R Kumar R K Aggarwal and J D Sharma ldquoEnergy analysis ofa building using artificial neural network A reviewrdquo Energy andBuildings vol 65 pp 352ndash358 2013

[6] C Alippi C De Russis and V Piuri ldquoA neural-network basedcontrol solution to air-fuel ratio control for automotive fuel-injection systemsrdquo IEEE Transactions on Systems Man andCybernetics Part C Applications and Reviews vol 33 no 2 pp259ndash268 2003

[7] E M Azoff Neural network time series forecasting of financialmarkets John Wiley amp Sons Inc 1994

[8] I Fister P N Suganthan J Fister et al ldquoArtificial neural net-work regression as a local search heuristic for ensemble strate-gies in differential evolutionrdquo Nonlinear Dynamics vol 84 no2 pp 895ndash914 2016

[9] Y Zhang S Li and H Guo ldquoA type of biased consensus-based distributed neural network for path planningrdquoNonlinearDynamics vol 89 no 3 pp 1803ndash1815 2017

[10] X Shi Z Wang and L Han ldquoFinite-time stochastic synchro-nization of time-delay neural networks with noise disturbancerdquoNonlinear Dynamics vol 88 no 4 pp 2747ndash2755 2017

[11] P He and Y Li ldquoHinfin synchronization of coupled reaction-diffusion neural networks with mixed delaysrdquo Complexity vol21 no S2 pp 42ndash53 2016

[12] Z Tu J Cao A Alsaedi F E Alsaadi and T Hayat ldquoGlobalLagrange stability of complex-valued neural networks of neutraltype with time-varying delaysrdquo Complexity vol 21 no S2 pp438ndash450 2016

[13] C Wang S Guo and Y Xu ldquoFormation of autapse connectedto neuron and its biological functionrdquo Complexity vol 2017Article ID 5436737 9 pages 2017

[14] J D J Rubio I Elias D R Cruz and J Pacheco ldquoUniform stableradial basis function neural network for the prediction in twomechatronic processesrdquo Neurocomputing vol 227 pp 122ndash1302017

[15] J d Rubio ldquoUSNFIS Uniform stable neuro fuzzy inferencesystemrdquo Neurocomputing vol 262 pp 57ndash66 2017

[16] Q Liu J Yin V C M Leung J-H Zhai Z Cai and J LinldquoApplying a new localized generalization error model to designneural networks trained with extreme learning machinerdquo Neu-ral Computing and Applications pp 1ndash8 2014

[17] R J de Jesus ldquoInterpolation neural network model of a manu-factured wind turbinerdquoNeural Computing and Applications pp1ndash12 2016

[18] C Mu and D Wang ldquoNeural-network-based adaptive guaran-teed cost control of nonlinear dynamical systems with matcheduncertaintiesrdquo Neurocomputing vol 245 pp 46ndash54 2017

[19] Z Lin DMa J Meng and L Chen ldquoRelative ordering learningin spiking neural network for pattern recognitionrdquo Neuro-computing 2017

[20] J Yu J Sang and X Gao ldquoMachine learning and signal pro-cessing for big multimedia analysisrdquo Neurocomputing vol 257pp 1ndash4 2017

Complexity 11

[21] T Zhang S S Ge and C C Hang ldquoAdaptive neural networkcontrol for strict-feedback nonlinear systems using backstep-ping designrdquo Automatica vol 36 no 12 pp 1835ndash1846 2000

[22] S S Ge and T Zhang ldquoNeural-network control of nonaffinenonlinear system with zero dynamics by state and outputfeedbackrdquo IEEE Transactions on Neural Networks and LearningSystems vol 14 no 4 pp 900ndash918 2003

[23] S S Ge C Yang and T H Lee ldquoAdaptive predictive controlusing neural network for a class of pure-feedback systems in dis-crete timerdquo IEEE Transactions on Neural Networks and LearningSystems vol 19 no 9 pp 1599ndash1614 2008

[24] F W Lewis S Jagannathan and A Yesildirak Neural networkcontrol of robotmanipulators and non-linear systems CRCPress1998

[25] S Jagannathan and F L Lewis ldquoIdentification of nonlineardynamical systems using multilayered neural networksrdquo Auto-matica vol 32 no 12 pp 1707ndash1712 1996

[26] D Vrabie and F Lewis ldquoNeural network approach tocontinuous-time direct adaptive optimal control for partiallyunknown nonlinear systemsrdquo Neural Networks vol 22 no 3pp 237ndash246 2009

[27] C Yang S S Ge and T H Lee ldquoOutput feedback adaptive con-trol of a class of nonlinear discrete-time systems with unknowncontrol directionsrdquo Automatica vol 45 no 1 pp 270ndash2762009

[28] C Yang Z Li and J Li ldquoTrajectory planning and optimizedadaptive control for a class of wheeled inverted pendulumvehicle modelsrdquo IEEE Transactions on Cybernetics vol 43 no 1pp 24ndash36 2013

[29] Y Jiang C Yang and H Ma ldquoA review of fuzzy logic andneural network based intelligent control design for discrete-time systemsrdquo Discrete Dynamics in Nature and Society ArticleID 7217364 Art ID 7217364 11 pages 2016

[30] Y Jiang C Yang S-L Dai and B Ren ldquoDeterministic learningenhanced neutral network control of unmanned helicopterrdquoInternational Journal of Advanced Robotic Systems vol 13 no6 pp 1ndash12 2016

[31] Y Jiang Z Liu C Chen and Y Zhang ldquoAdaptive robust fuzzycontrol for dual arm robot with unknown input deadzonenonlinearityrdquo Nonlinear Dynamics vol 81 no 3 pp 1301ndash13142015

[32] httpsstemcellthailandorgneurons-definition-function-neurotransmitters

[33] MDefoort T Floquet A Kokosy andW Perruquetti ldquoSliding-mode formation control for cooperative autonomous mobilerobotsrdquo IEEE Transactions on Industrial Electronics vol 55 no11 pp 3944ndash3953 2008

[34] X Liu C Yang Z Chen MWang and C Su ldquoNeuro-adaptiveobserver based control of flexible joint robotrdquoNeurocomputing2017

[35] F Hamerlain T Floquet and W Perruquetti ldquoExperimentaltests of a slidingmode controller for trajectory tracking of a car-like mobile robotrdquo Robotica vol 32 no 1 pp 63ndash76 2014

[36] R J de Jesus ldquoDiscrete time control based in neural networksfor pendulumsrdquo Applied Soft Computing 2017

[37] Y Pan M J Er T Sun B Xu and H Yu ldquoAdaptive fuzzy PDcontrol with stable Hinfin tracking guaranteerdquo Neurocomputingvol 237 pp 71ndash78 2017

[38] R J de Jesus ldquoAdaptive least square control in discrete time ofrobotic armsrdquo Soft Computing vol 19 no 12 pp 3665ndash36762015

[39] S Commuri S Jagannathan and F L Lewis ldquoCMAC neuralnetwork control of robot manipulatorsrdquo Journal of RoboticSystems vol 14 no 6 pp 465ndash482 1997

[40] J S Albus ldquoTheoretical and experimental aspects of a cerebellarmodelrdquoDevelopmental Disabilities Research Reviews vol 17 pp93ndash101 1972

[41] B Yang R Bao and H Han ldquoRobust hybrid control based onPD and novel CMAC with improved architecture and learningscheme for electric load simulatorrdquo IEEE Transactions onIndustrial Electronics vol 61 no 10 pp 5271ndash5279 2014

[42] S Jagannathan and F L Lewis ldquoMultilayer discrete-timeneural-net controller with guaranteed performancerdquo IEEETransactions on Neural Networks and Learning Systems vol 7no 1 pp 107ndash130 1996

[43] S S Ge and J Wang ldquoRobust adaptive neural control for a classof perturbed strict feedback nonlinear systemsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no6 pp 1409ndash1419 2002

[44] Y H Kim F L Lewis and C T Abdallah ldquoA dynamic recurrentneural-network-based adaptive observer for a class of nonlinearsystemsrdquo Automatica vol 33 no 8 pp 1539ndash1543 1997

[45] J-Q Huang and F L Lewis ldquoNeural-network predictive controlfor nonlinear dynamic systems with time-delayrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 14 no 2pp 377ndash389 2003

[46] S S Ge F Hong and T H Lee ldquoAdaptive neural control ofnonlinear time-delay systems with unknown virtual controlcoefficientsrdquo IEEE Transactions on Systems Man and Cybernet-ics Part B Cybernetics vol 34 no 1 pp 499ndash516 2004

[47] J Na X M Ren and H Huang ldquoTime-delay positive feedbackcontrol for nonlinear time-delay systems with neural networkcompensationrdquo Zidonghua XuebaoActa Automatica Sinica vol34 no 9 pp 1196ndash1202 2008

[48] J Na X Ren C Shang and Y Guo ldquoAdaptive neural networkpredictive control for nonlinear pure feedback systems withinput delayrdquo Journal of Process Control vol 22 no 1 pp 194ndash206 2012

[49] R R Selmic and F L Lewis ldquoNeural-network approximationof piecewise continuous functions application to friction com-pensationrdquo IEEE Transactions on Neural Networks and LearningSystems vol 13 no 3 pp 745ndash751 2002

[50] H Zhang and F L Lewis ldquoAdaptive cooperative trackingcontrol of higher-order nonlinear systems with unknowndynamicsrdquo Automatica vol 48 no 7 pp 1432ndash1439 2012

[51] J Na X Ren and D Zheng ldquoAdaptive control for nonlinearpure-feedback systems with high-order sliding mode observerrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 24 no 3 pp 370ndash382 2013

[52] J Na Q Chen X Ren and Y Guo ldquoAdaptive prescribedperformancemotion control of servomechanisms with frictioncompensationrdquo IEEE Transactions on Industrial Electronics vol61 no 1 pp 486ndash494 2014

[53] J Na X Ren G Herrmann and Z Qiao ldquoAdaptive neuraldynamic surface control for servo systems with unknown dead-zonerdquoControl Engineering Practice vol 19 no 11 pp 1328ndash13432011

[54] G Li J Na D P Stoten and X Ren ldquoAdaptive neural networkfeedforward control for dynamically substructured systemsrdquoIEEE Transactions on Control Systems Technology vol 22 no3 pp 944ndash954 2014

Complexity 5

Action network

Modelnetwork

Critic network

Critic network

x(k)u(k)

x(k + 1) U(k)

J(x(k))

J(x(k + 1))

minus

Figure 4 An overview of the HDP structure

systems in the presence of input constraints By using this RL-based controller a constrained optimal control problem wassolved with construction of only one critic neural network In[69] an ADP technique for online control and learning of ageneralized multiple-input-multiple-output (MIMO) systemwas investigated In [70] an adaptive NN based ADP controlscheme was presented for a class of nonlinear systems withunknown dynamics The optimal control law was calculatedby using a dual neural network scheme with a critic NN andan identifier NN Particularly parameters estimation errorwas used to online identify the learning weights to achievethe finite-time convergence Optimal tracking control for aclass of nonlinear systems was investigated in [71] where anew ldquoidentifier-criticrdquo based ADP framework was proposed

33 Evolutionary Computing In addition to the capacity ofapproximation and optimization of the NN there has beenalso a great interest in using the evolutionary approachesto train the neural networks With the evolvement of NNarchitectures learning rules connection weights and inputfeatures an evolutionary artificial neural network (EANN)was designed to provide superior performance in comparisonto conventional training approaches [72] A literature reviewof the EANNwas given in [73] where the evolution strategiessuch as feedforward artificial NN and genetic algorithms(GA) have been introduced for the EANNs In [72] severalEANN frameworks were introduced by embedding the evo-lution algorithms (EA) to evolve the NN structure In [74]an EPNet evolution system was proposed for evolving thefeedforward NN based on Fogelrsquos evolutionary programming(EP) method which could improve the NNrsquos connectionweights and architectures at the same time as well as decreasenoise in the fitness evaluation Good generalization abilityof the evolved NN has been constructed and verified inthe experiments In [75] a GA based technique has beenemployed to train the NNs in direct neural control systemssuch that the NN architectures could be optimized Adeficiency of the EANN is that the optimization processwould often result in a low training speed To overcomethis problem and facilitate adaptation processes a hybridmultiobjective evolutionary method was developed in [76]where the singular-value-decomposition (SVD) techniquewas employed to choose the necessary neurons number in thetraining of a feedforwardNNThe evolutionary approachwas

applied to identify a grey-box model with a multiobjectiveoptimization between the clearly known practical systemsand approximated nonlinear systems [77] Applications ofevolutionary algorithms for robotic navigation have beenintroduced and investigated in [78] A survey of machinelearning technique was reported in [79] where several meth-ods to improve the evolutionary computation were reviewed

The evolution algorithms have been employed in manyaspects for evolvements of NNs such as to train the NN con-nection weights or to obtain near-optimal NN architecturesas well as adapting learning rules of NNs to their environ-ment In a word the evolution algorithms provide NNs withthe ability of learning to learn and also to build the relation-ship between evolution and learning such that the EANNcould perform favorable ability to adapt to changes of thedynamic environment

4 Applications in Robots

41 NNBased RoboticManipulator Control Generally speak-ing the control methods for robot manipulators can beroughly divided into two groups model-free control andmodel based control For the model-free control approacheslike proportional-integral-derivative (PID) control satisfac-tory control performancemay not be guaranteed In contrastthe model based control approaches exhibit better controlbehavior but heavily depend on the validity of the robotmodel In practice however a perfect robotic dynamicmodel is always not available due to the complex mech-anisms and uncertainties Additionally the payload maybe varied according to different tasks which makes theaccurate dynamics model hard to be obtained in advance Tosolve such problems the NN approximation-based controlmethods have been used extensively in applications of robotmanipulator control A basic structure of the adaptive neuralnetwork control for robot manipulator is shown in Figure 5Consider a dynamic model of a robot manipulator given asfollows [80]

119872(119902) 119902 + 119862 ( 119902 119902) 119902 + 119866 (119902) = 120591 (9)

where119872(119902)119862( 119902 119902) and119866(119902) are the inertialmatrix Coriolismatrix and gravity vector respectively Then NN controldesign could be given as follows

120591119889 = minus11987011198901 minus 11987021198902 + 119878 (119885) (10)

6 Complexity

PD controller Robotic arm

Closed loop neural network controller

Neural network controller

Neuralinputs

qd qd e e q q q1

d

Figure 5 A basic structure of the adaptive NN robot control

where 1198901 is the tracking error 1198902 is the velocity tracking error119878(119885) is the NN controller with being the weights matrixand 119878(119885) being the NN regressor vector and 1198701 and 1198702 arecontrol gains specified by the designer

From (10) we can see that the robot controller consists ofa PD-like controller and aNN controller In traditionalmodelbased controllers the dynamic model of the robot could beregarded as a feedforward to address the effect caused bythe robot motion In practice however 119872(119902) 119862(119902 119902) and119866(119902) may not be known Therefore the NNs are used toapproximate the unknown dynamics 119891 = 119872(119902) 119902 +119862( 119902 119902) 119902 +119866(119902) and to improve the performance of the system via theonline estimation To adapt theNNweights adaptive laws aredesigned as follows

119882 = Γ (119878119890 minus 120590) (11)

where Γ and 120590 are specified positive parametersThe last termof right-hand side of (11) is the sigma modification whichis used to enhance the convergence and robustness of theparameters adaptation

In [80] a NN based share control method was devel-oped to control a teleoperated robot with environmentaluncertainties In this work the RBFNN was constructed tocompensate for the unknown dynamics of the teleoperatedrobot Particularly a shared control strategy was developedinto the controller to achieve the automatic obstacle avoid-ance combining with the information of visual camera andthe robot body such that the obstacle could be successfullyavoided and the operator could focus more on the operatedtask rather than the environment to guarantee the stabilityand manipulation In addition error transformations wereintegrated into the adaptiveNN control to guarantee the tran-sient control performance It was shown that by using theNNtechnique the control performance in both kinematic leveland dynamic level of the teleoperated robot was enhancedIn [81] an extreme learning machine (ELM) based controlstrategy was proposed for uncertain robot manipulators toidentify both the elasticity and geometry of an object ThisELM was applied to deal with the unknown nonlinearity ofthe robot manipulator to enhance the control performanceParticularly by utilizing ELM the proposed controller couldguarantee that the robot dynamics follow a reference modelsuch that the desired set point and the feedforward forcecould be updated to estimate the geometry and stiffness ofthe object As a result the reference model could be exactlymatched with a limited number of iterations

In [82] the NN controller was also employed to control awheel inverted pendulum which has been decomposed intotwo subsystems a fully actuated second-order planar movingsubsystem and a passive first-order pendulum subsystemThen the RBFNN was employed to compensate for theuncertain dynamics of the two subsystems by using itspowerful learning ability such that the enhanced controlperformance could be realized by using the NN learning In[83] a global adaptive neural control was proposed for a classof robot manipulators with finite-time convergence learn-ing performance This control scheme employed a smoothswitching mechanism combining with a nominal neuralnetwork controller and a robust controller to ensure globaluniform ultimately bounded stability The optimal weightswere obtained by the finite-time estimation algorithm suchthat after the learning process the learning weights could bereused next time for repeated tasks The global NN controlmechanism has been further extended to the control of dualarm robot manipulator in [84] where knowledge of bothrobot manipulator and the grasping object is unavailable inadvance By integrating prescribed functions into the designof controller the transient performance of the dual armrobot control was regularly guaranteed The NN was alsoemployed to deal with synchronization problem of multiplerobot manipulators in [85] where the reference trajectoriesare only available for part of the team members By usingthe NN approximation controller the robot has shown bettercontrol performance with enhanced transient performanceand enhanced robustness A RBFNN was constructed tocompensate for the nonlinear terms of a five-bar manipulatorbased on an error transformation function [86] The NNcontrol was also applied in the robot teleoperation control[87 88] Moreover a NN approximation technique wasemployed to deal with the unknown dynamics kinematicsand actuator properties in the manipulator tracking control[89]

42 NN Based Robot Control with Input NonlinearitiesAnother challenge of the robot manipulator is that theinput nonlinearities such as friction dead-zone and actuatorsaturation may inevitably exist in the robot systems Theseinput nonlinearities may lead to larger tracking errors anddegeneration of the control performance Therefore a num-ber of works have been proposed to handle the nonlinearitiesby utilizing the neural network design A neural adaptive

Complexity 7

controller was designed to deal with the effect of inputsaturation of the robot manipulator in [90] as follows

120591 = minus1199111 + 119863119878119863 (119885119863) 1205721 + 119862119878119862 (119885119862) 1205721+ 119866119878119866 (119885119866) + 119870119901 (1199112 + 120585) + 119870119903 sgn (1199112) (12)

where 1199111 is the robot position tracking error 1199112 is the velocitytracking error and 1205721 is an auxiliary controller 119863 119862 and119866 are the NN weights 119878119863(119885119863) 119878119862(119885119862) and 119878119866(119885119866) arethe NN regressor vectors and 119870119901 and 119870119903 are control gainsspecified by the designer 120585 is an auxiliary system designed toreduce the effect of the saturation with 120585 defined as follows

120585

= minus119870120585120585 minus

100381610038161003816100381610038161199111198792Δ12059110038161003816100381610038161003816 + (12) Δ120591119879Δ120591100381710038171003817100381712058510038171003817100381710038172 120585 + Δ120591 10038171003817100381710038171205851003817100381710038171003817 ge 1205830 10038171003817100381710038171205851003817100381710038171003817 lt 120583

(13)

where Δ120591 is the torque error caused by saturation and119870120585 is asmall positive value To update the NNweights adaptive lawsare designed as follows

119882119863 = Γ119863119896 (11987811986311989612057211198902119896 minus 120590119863119896119863119896)119882119862 = Γ119862119896 (11987811986211989612057211198902119896 minus 120590119862119896119862119896)119882119866 = Γ119866119896 (11987811986611989612057211198902119896 minus 120590119866119896119866119896)

(14)

where Γ119863119896 Γ119862119896 and Γ119866119896 are specified positive parameters and120590119863119896 120590119862119896 and 120590119866119896 are positive parametersIn [91] an adaptive neural network controller was con-

structed to approximate the input dead-zone and the uncer-tain dynamics of the robotic manipulator while the outputconstraint was also considered in the feedback control In[92] the NN was applied for the estimation of the unknownmodel parameters of a marine surface vessel and in [93] thefull-state constraint of an n-link robotic manipulator wasachieved by using the NN control The NN controller wasalso constructed for flexible roboticmanipulators to deal withthe vibration suppression based on a lumped spring-massmodel [94] while in [95] two RBFNNs were constructed forflexible robot manipulators to compensate for the unknowndynamics and the dead-zone effect respectively

The NN has also been used in many important industrialfields such as autonomous underwater vehicles (AUVs) andhypersonic flight vehicle (HFV) In [96] the NN has beenconstructed to deal with the attitude of AUVs in the presenceof input dead-zone and uncertain model parameters In [97]the adaptive neural control was employed to deal with under-water vehicle control in discrete-time domain encounteredwith the unknown input nonlinearities external disturbanceand model uncertainties Then the reinforcement learningwas applied to address these uncertainties by using a criticNN and an action NN The hypersonic flight vehicle controlwas investigated in [98] where the aerodynamic uncertaintiesand unknown disturbances were addressed by a disturbance

observer based NN In [99] a neural learning control wasembedded in the HFV controller to achieve the globalstability via a switching mechanism and a robust controller

43 NN Based Human-Robot Interaction Control Recentlythere is a predominant tendency to employ the robots inthe human-surrounded environment such as householdservices or industrial applications where humans and robotsmay interact with each other directly Therefore interactioncontrol has become a promising research field and has beenwidely studied In [100] a learning method was developedsuch that the dynamics of a robot arm could follow atarget impedance model with only knowledge of the roboticstructure (see Figure 6)

TheNNwas further employed in robot control in interac-tionwith an environment [101] where impedance control wasachieved with the completely unknown robotic dynamicsIn [102] a learning method was developed such that therobot was able to adjust the impedance parameters when itinteracted with unknown environments In order to learnoptimal impedance parameters in the robot manipulatorcontrol an adaptive dynamic programming (ADP) methodwas employed when the robot interacted with unknowntime-varying environments where NNs were used for bothcritic and actor networks [103] The ADP was also employedfor coordination of multirobots [104] in which possibledisagreement between different manipulators was handledand dynamics of both robots and themanipulated object werenot required to be known

In this work the controller consists of two parts a criticnetwork which was used to approximate the cost functionand an actual NN which was designed to control the robotThe critic NN is designed as follows [104]

Υ (119905) = 119862119878119862 (119885119862) (15)

where 119885119862 = 120585 120585 = [119879119900 119911119879 119909119879119900 ]119879 with 119909119900 being the positionof the object and 119911 being the tracking error 119862 is the NNweight and 119878119862 is the regressor vector

The critic NN is used to approximate a cost function119888(119905) = 120585119879119876120585 + 119906119879119877119906 where 119906 denotes the control inputand 119876 and 119877 are positive definite matrix Since the controlobjective is to minimize the control effort the adaptation lawis designed as

119882119888 = minus120590119888 120597119864119888120597119888 = 120590119888 (119888 minus 119879119888 119878119862) 119878119862 (16)

where 120590119888 is the learning rate and 119864119888 = (12)(119888 minus 119879119888 119878119862)2On the other hand the actual NN control is designed to

control the robot as

119906 = 119879119886 119878119886 (119885119886) minus 119890 minus 1198702119890V (17)

where 119879119886 119878119886(119885119886) could learn the dynamics of the robotwith 119886 being the NN weight and 119878119886 being the regressorvector 119890 and 119890V are the position and velocity tracking errorsrespectively and 1198702 is the control gain

8 Complexity

Positon control Robotic arm Inverse

kinematics

Forward kinematics

Optimal critic learning

EnvironmentImpedance

model

xd

xrqr

x q

f

minus

+

Figure 6 Block of the learning impedence control

Since the control objective is to guarantee the estimationof both robot dynamics and the cost function Υ(119905) theadaptive law is selected as follows

119882119879119886119894 = minus120590119886 (119879119886119894119878119886 + 119896119903Υ) 119878119886 (18)

where 120590119886 and 119896119903 are positive constantsOn the other hand as a fundamental element of the next-

generation robots the human-robot collaboration (HRC) hasbeen widely studied by roboticists and NN is employed inHRCwith its powerful learning ability In [105] theNNswereemployed to estimate the human partnerrsquos motion intentionin human-robot collaboration such that the robot was able toactively follow its human partner To adjust the robotrsquos role tolead or to follow according to the humanrsquos intention gametheory was employed for fundamental analysis of human-robot interaction and an adaptation law was developed in[106] Policy iteration combining with NN was adopted toprovide a rigorous solution to the problem of the systemequilibrium in human-robot interaction [107]

44 NN Based Robot Cognitive Control According to thepredictive processing theory [108] the human brain is alwaysactively anticipating the incoming sensorimotor informationThis process exists because the living beings exhibit latenciesdue to neural processing delays and a limited bandwidthin their sensorimotor processing To compensate for sucha delay in human brain neural feedback signals (includinglateral and top-down connections)modulate the neural activ-ities via inhibitory or excitatory connections by influencingthe neuronal population coding of the bottom-up sensory-driven signals in the perception-action system Similarlyin robotic systems it is claimed that such a delay and alimited bandwidth also can be compensated by the predictivefunctions learnt by recurrent neural models Such a learningprocess can be done via only visual processing [109] or in theloop of perception and action [110]

Based on the hierarchical sensorimotor integration the-ory which advocates that action and perception are inter-twined by sharing the same representational basis [111] therepresentation on different levels of sensory perception doesnot explicitly represent actions instead there is an encodingof the possible future percept which is learnt from priorsensorimotor knowledge

In the Bayesian once this perception and action linkshave been established after learning these perception-action

associations in this architecture allow the following opera-tions

First these associations allow predicting the perceptualoutcome of given actions by means of the forward models(eg Bayesian model) It can be written as

119875 (119864 | 119860 119868) prop 119875 (119860 | 119864) 119875 (119864 | 119868) (19)

where E estimates the upcoming perception evidence givenan executed action A and other prior information you havealready known in 119868 The term 119875(119864 | 119860 119868) suggests that aprelearntmodel representing the possibility of amotor actionA will be executed given that a (possible) resulting sensoryevidence 119864 is perceived (backward computation)

Second these associations allow selecting an appropri-ate movement given an intended perceptual representationFrom the backward computations introduced in the followingequation a predictive sensorimotor integration occurs

119875 (119860 | 119864 119866) prop 119875 (119864 | 119860) 119875 (119860 | 119866) (20)

where A indicates a particular action selected given the(intended) sensory information 119864 and a goal G Here weassume that onersquos action is only determined by the currentsensory input and the goal

In terms of its hierarchical organization it also allows thisoperation with bidirectional information pathways a lowlevel perception representation can be expressed on a higherlevel with a more complex receptive field and vice versa119890low hArr 119890high This can be realized by the bidirectional deeparchitectures such as [112] Conceptually these operationscan be achieved by extracting statistical regularity shown inFigure 7

Since both perception and action processes can be seenas temporal sequences from the mathematical perspectivethe recurrent networks are Turing-Complete and have alearning capacity to learn time sequences with arbitrarylength [113] if properly trained Furthermore such recurrentconnections can be placed in a hierarchical way in whichthe prediction functions on different layers attempt to predictthe nonlinear time-series in different time-scales [114] Fromthis point the recurrent neural network with parametric biasunits (RNNPB) [115] andmultiple time-scale recurrent neuralnetworks (MTRNN) [116] were applied to predict sequencesby understanding them in various temporal levels

The difference of the temporal levels controls the prop-erties of the different levels of the presentation in the deep

Complexity 9

SMA

M1

Brainstem

Spine

Action

Common coding

Common coding

MST

V5MT

V3

V1

Cognitiveprocesses

Visual perception(dorsal)

Figure 7 The process of cognitive control

recurrent network For instance in the MTRNN network[112] the learning of each neuron follows the updating rule ofclassical firing rate models in which the activity of a neuronis determined by the average firing rate of all the connectedneurons Additionally the neuronal activity is also decayingover time following an updating rule of leaky integratormodel Assuming the i-th MTRNN neuron has the numberof N connections the current membrane potential status ofa neuron can be defined by both the previous activation andthe current synaptic inputs

119906119894119905+1 = (1 minus 1120591119894)119906119894119905 + 1120591119894 [sum119908119894119895119909119895119905] (21)

where 119908119894119895 represents the synaptic weight from the j-thneuron to the i-th neuron 119909119895119905 is the activity of j-th neuronat t-th time-step and 120591 is the time-scale parameter whichdetermines the decay rate of this neuron a larger 120591 meanstheir activities change slowly over time compared with thosewith a smaller time-scale parameter 120591

In [117] the concepts of predictive coding were discussedin detail where the learning generation and recognition ofactions can be conducted by means of the principle of pre-diction error minimization By using the predictive codingthe RNNPB and MTRNN are capable for both generatingown actions and recognizing the same actions performedby others Recently the study on neurorobotics experimentshas shown that the dynamic predictive coding scheme canbe used to address fluctuations in temporal patterns whentraining a recurrent neural network (RNN) model [118]This predictive coding scheme enables organisms to predictperceptual outcomes based on current intentions of actionsto the external environment and to forecast perceptualsequences corresponding to given intention states [118]

Based on this architecture two-layer RNN models wereutilized to extract visual information [119] and to under-stand intentions [120] or emotion status [121] in socialrobotics three-layer RNNmodels were used to integrate and

understand multimodal information for a humanoid iCubrobot [112 122] Moreover the predictive coding frameworkhas been extended to variational Bayes predictive codingMTRNN which can arbitrate between deterministic modeland probabilistic model by setting a metaparameter [123]Such extension could provide significant improvement indealing with noisy fluctuated sensory inputs which robots areexpected to experience in more real world setting In [124]a MTRNN was employed to control a humanoid robot andexperimental results have shown that by using only partialtraining data the control model can achieve generalizationby learning in a lower feature perception level

The hierarchical structure of RNN exhibits a greatlearning capacity to store multimodal information whichis beneficial for the robotic systems to understand and topredict in a complex environment As the future models andapplications the state-of-the-art deep learning techniques orthemotor actions of robotic systems can be further integratedinto this predictive architecture

5 Conclusion

In summary great achievements for control design of nonlin-ear system by means of neural networks have been gained inthe last two decades Despite the impossibility in identifyingor listing all the related contributions in this short reviewefforts have been made to summarize the recent progressin the area of NN control and its particular applications inthe robot learning control the robot interaction control andthe robot recognition control In this paper we have shownthat significant progress of NN has been made in control ofthe nonlinear systems in solving the optimization problemin approximating the system dynamics in dealing with theinput nonlinearities in human-robot interaction and in thepattern recognition All these developments accompany notonly the development of techniques in control and advancedmanufactures but also theatrical progress in constructingand developing the neural networks Although huge efforts

10 Complexity

have been made to embed the NN in practical control sys-tems there is still a large gap between the theory and practiceTo improve the feasibility and usability the evolutionarycomputing theory has been proposed to train the NNs It canautomatically find a near-optimal NN architecture and allowa NN to adapt its learning rule to its environment Howeverthe complex and long training process of the evolutionaryalgorithms deters their practical applications More effortsneed to be made to evolve the NN architecture and NNlearning technique in the control design On the otherhand in human brain the neural activities are modulatedvia inhibitory or excitatory connections by influencing theneuronal population coding of the bottom-up sensory-drivensignals in the perception-action system In this sense howto integrate the sensor-motor information into the networkto make NNs more feasible to adapt to the environment andto resemble the capacity of the human brain deserves furtherinvestigations

In conclusion a brief review on neural networks for thecomplex nonlinear systems is provided with adaptive neuralcontrol NN based dynamic programming evolution com-puting and their practical applications in the robotic fieldsWe believe this area may promote increasing investigationsin both theories and applications And emerging topics likedeep learning [125ndash128] big data [129ndash131] and cloud com-puting may be incorporated into the neural network controlfor complex systems for example deep neural networkscould be used to process massive amounts of unsuperviseddata in complex scenarios neural networks can be helpfulin reducing the data dimensionality and the optimization ofNN training may be employed to enhance the learning andadaptation performance of robots

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

Thisworkwas partially supported by theNational Nature Sci-ence Foundation (NSFC) under Grant 61473120 GuangdongProvincial Natural Science Foundation 2014A030313266International Science and Technology Collaboration Grant2015A050502017 Science andTechnology Planning Project ofGuangzhou 201607010006 State Key Laboratory of Roboticsand System (HIT) Grant SKLRS-2017-KF-13 and the Funda-mental Research Funds for the Central Universities

References

[1] F Gruau D Whitley and L Pyeatt ldquoA comparison betweencellular encoding and direct encoding for genetic neural net-worksrdquo inProceedings of the 1st annual conference on genetic pro-gramming pp 81ndash89 1996

[2] H de Garis ldquoAn artificial brain ATRrsquos CAM-Brain Project aimsto buildevolve an artificial brain with a million neural netmodules inside a trillion cell Cellular Automata Machinerdquo NewGeneration Computing vol 12 no 2 pp 215ndash221 1994

[3] W S McCulloch and W Pitts ldquoA logical calculus of the ideasimmanent in nervous activityrdquoBulletin ofMathematical Biologyvol 5 no 4 pp 115ndash133 1943

[4] C Yang S S Ge C Xiang T Chai and T H Lee ldquoOutput feed-back NN control for two classes of discrete-time systems withunknown control directions in a unified approachrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 19 no 11pp 1873ndash1886 2008

[5] R Kumar R K Aggarwal and J D Sharma ldquoEnergy analysis ofa building using artificial neural network A reviewrdquo Energy andBuildings vol 65 pp 352ndash358 2013

[6] C Alippi C De Russis and V Piuri ldquoA neural-network basedcontrol solution to air-fuel ratio control for automotive fuel-injection systemsrdquo IEEE Transactions on Systems Man andCybernetics Part C Applications and Reviews vol 33 no 2 pp259ndash268 2003

[7] E M Azoff Neural network time series forecasting of financialmarkets John Wiley amp Sons Inc 1994

[8] I Fister P N Suganthan J Fister et al ldquoArtificial neural net-work regression as a local search heuristic for ensemble strate-gies in differential evolutionrdquo Nonlinear Dynamics vol 84 no2 pp 895ndash914 2016

[9] Y Zhang S Li and H Guo ldquoA type of biased consensus-based distributed neural network for path planningrdquoNonlinearDynamics vol 89 no 3 pp 1803ndash1815 2017

[10] X Shi Z Wang and L Han ldquoFinite-time stochastic synchro-nization of time-delay neural networks with noise disturbancerdquoNonlinear Dynamics vol 88 no 4 pp 2747ndash2755 2017

[11] P He and Y Li ldquoHinfin synchronization of coupled reaction-diffusion neural networks with mixed delaysrdquo Complexity vol21 no S2 pp 42ndash53 2016

[12] Z Tu J Cao A Alsaedi F E Alsaadi and T Hayat ldquoGlobalLagrange stability of complex-valued neural networks of neutraltype with time-varying delaysrdquo Complexity vol 21 no S2 pp438ndash450 2016

[13] C Wang S Guo and Y Xu ldquoFormation of autapse connectedto neuron and its biological functionrdquo Complexity vol 2017Article ID 5436737 9 pages 2017

[14] J D J Rubio I Elias D R Cruz and J Pacheco ldquoUniform stableradial basis function neural network for the prediction in twomechatronic processesrdquo Neurocomputing vol 227 pp 122ndash1302017

[15] J d Rubio ldquoUSNFIS Uniform stable neuro fuzzy inferencesystemrdquo Neurocomputing vol 262 pp 57ndash66 2017

[16] Q Liu J Yin V C M Leung J-H Zhai Z Cai and J LinldquoApplying a new localized generalization error model to designneural networks trained with extreme learning machinerdquo Neu-ral Computing and Applications pp 1ndash8 2014

[17] R J de Jesus ldquoInterpolation neural network model of a manu-factured wind turbinerdquoNeural Computing and Applications pp1ndash12 2016

[18] C Mu and D Wang ldquoNeural-network-based adaptive guaran-teed cost control of nonlinear dynamical systems with matcheduncertaintiesrdquo Neurocomputing vol 245 pp 46ndash54 2017

[19] Z Lin DMa J Meng and L Chen ldquoRelative ordering learningin spiking neural network for pattern recognitionrdquo Neuro-computing 2017

[20] J Yu J Sang and X Gao ldquoMachine learning and signal pro-cessing for big multimedia analysisrdquo Neurocomputing vol 257pp 1ndash4 2017

Complexity 11

[21] T Zhang S S Ge and C C Hang ldquoAdaptive neural networkcontrol for strict-feedback nonlinear systems using backstep-ping designrdquo Automatica vol 36 no 12 pp 1835ndash1846 2000

[22] S S Ge and T Zhang ldquoNeural-network control of nonaffinenonlinear system with zero dynamics by state and outputfeedbackrdquo IEEE Transactions on Neural Networks and LearningSystems vol 14 no 4 pp 900ndash918 2003

[23] S S Ge C Yang and T H Lee ldquoAdaptive predictive controlusing neural network for a class of pure-feedback systems in dis-crete timerdquo IEEE Transactions on Neural Networks and LearningSystems vol 19 no 9 pp 1599ndash1614 2008

[24] F W Lewis S Jagannathan and A Yesildirak Neural networkcontrol of robotmanipulators and non-linear systems CRCPress1998

[25] S Jagannathan and F L Lewis ldquoIdentification of nonlineardynamical systems using multilayered neural networksrdquo Auto-matica vol 32 no 12 pp 1707ndash1712 1996

[26] D Vrabie and F Lewis ldquoNeural network approach tocontinuous-time direct adaptive optimal control for partiallyunknown nonlinear systemsrdquo Neural Networks vol 22 no 3pp 237ndash246 2009

[27] C Yang S S Ge and T H Lee ldquoOutput feedback adaptive con-trol of a class of nonlinear discrete-time systems with unknowncontrol directionsrdquo Automatica vol 45 no 1 pp 270ndash2762009

[28] C Yang Z Li and J Li ldquoTrajectory planning and optimizedadaptive control for a class of wheeled inverted pendulumvehicle modelsrdquo IEEE Transactions on Cybernetics vol 43 no 1pp 24ndash36 2013

[29] Y Jiang C Yang and H Ma ldquoA review of fuzzy logic andneural network based intelligent control design for discrete-time systemsrdquo Discrete Dynamics in Nature and Society ArticleID 7217364 Art ID 7217364 11 pages 2016

[30] Y Jiang C Yang S-L Dai and B Ren ldquoDeterministic learningenhanced neutral network control of unmanned helicopterrdquoInternational Journal of Advanced Robotic Systems vol 13 no6 pp 1ndash12 2016

[31] Y Jiang Z Liu C Chen and Y Zhang ldquoAdaptive robust fuzzycontrol for dual arm robot with unknown input deadzonenonlinearityrdquo Nonlinear Dynamics vol 81 no 3 pp 1301ndash13142015

[32] httpsstemcellthailandorgneurons-definition-function-neurotransmitters

[33] MDefoort T Floquet A Kokosy andW Perruquetti ldquoSliding-mode formation control for cooperative autonomous mobilerobotsrdquo IEEE Transactions on Industrial Electronics vol 55 no11 pp 3944ndash3953 2008

[34] X Liu C Yang Z Chen MWang and C Su ldquoNeuro-adaptiveobserver based control of flexible joint robotrdquoNeurocomputing2017

[35] F Hamerlain T Floquet and W Perruquetti ldquoExperimentaltests of a slidingmode controller for trajectory tracking of a car-like mobile robotrdquo Robotica vol 32 no 1 pp 63ndash76 2014

[36] R J de Jesus ldquoDiscrete time control based in neural networksfor pendulumsrdquo Applied Soft Computing 2017

[37] Y Pan M J Er T Sun B Xu and H Yu ldquoAdaptive fuzzy PDcontrol with stable Hinfin tracking guaranteerdquo Neurocomputingvol 237 pp 71ndash78 2017

[38] R J de Jesus ldquoAdaptive least square control in discrete time ofrobotic armsrdquo Soft Computing vol 19 no 12 pp 3665ndash36762015

[39] S Commuri S Jagannathan and F L Lewis ldquoCMAC neuralnetwork control of robot manipulatorsrdquo Journal of RoboticSystems vol 14 no 6 pp 465ndash482 1997

[40] J S Albus ldquoTheoretical and experimental aspects of a cerebellarmodelrdquoDevelopmental Disabilities Research Reviews vol 17 pp93ndash101 1972

[41] B Yang R Bao and H Han ldquoRobust hybrid control based onPD and novel CMAC with improved architecture and learningscheme for electric load simulatorrdquo IEEE Transactions onIndustrial Electronics vol 61 no 10 pp 5271ndash5279 2014

[42] S Jagannathan and F L Lewis ldquoMultilayer discrete-timeneural-net controller with guaranteed performancerdquo IEEETransactions on Neural Networks and Learning Systems vol 7no 1 pp 107ndash130 1996

[43] S S Ge and J Wang ldquoRobust adaptive neural control for a classof perturbed strict feedback nonlinear systemsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no6 pp 1409ndash1419 2002

[44] Y H Kim F L Lewis and C T Abdallah ldquoA dynamic recurrentneural-network-based adaptive observer for a class of nonlinearsystemsrdquo Automatica vol 33 no 8 pp 1539ndash1543 1997

[45] J-Q Huang and F L Lewis ldquoNeural-network predictive controlfor nonlinear dynamic systems with time-delayrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 14 no 2pp 377ndash389 2003

[46] S S Ge F Hong and T H Lee ldquoAdaptive neural control ofnonlinear time-delay systems with unknown virtual controlcoefficientsrdquo IEEE Transactions on Systems Man and Cybernet-ics Part B Cybernetics vol 34 no 1 pp 499ndash516 2004

[47] J Na X M Ren and H Huang ldquoTime-delay positive feedbackcontrol for nonlinear time-delay systems with neural networkcompensationrdquo Zidonghua XuebaoActa Automatica Sinica vol34 no 9 pp 1196ndash1202 2008

[48] J Na X Ren C Shang and Y Guo ldquoAdaptive neural networkpredictive control for nonlinear pure feedback systems withinput delayrdquo Journal of Process Control vol 22 no 1 pp 194ndash206 2012

[49] R R Selmic and F L Lewis ldquoNeural-network approximationof piecewise continuous functions application to friction com-pensationrdquo IEEE Transactions on Neural Networks and LearningSystems vol 13 no 3 pp 745ndash751 2002

[50] H Zhang and F L Lewis ldquoAdaptive cooperative trackingcontrol of higher-order nonlinear systems with unknowndynamicsrdquo Automatica vol 48 no 7 pp 1432ndash1439 2012

[51] J Na X Ren and D Zheng ldquoAdaptive control for nonlinearpure-feedback systems with high-order sliding mode observerrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 24 no 3 pp 370ndash382 2013

[52] J Na Q Chen X Ren and Y Guo ldquoAdaptive prescribedperformancemotion control of servomechanisms with frictioncompensationrdquo IEEE Transactions on Industrial Electronics vol61 no 1 pp 486ndash494 2014

[53] J Na X Ren G Herrmann and Z Qiao ldquoAdaptive neuraldynamic surface control for servo systems with unknown dead-zonerdquoControl Engineering Practice vol 19 no 11 pp 1328ndash13432011

[54] G Li J Na D P Stoten and X Ren ldquoAdaptive neural networkfeedforward control for dynamically substructured systemsrdquoIEEE Transactions on Control Systems Technology vol 22 no3 pp 944ndash954 2014

6 Complexity

PD controller Robotic arm

Closed loop neural network controller

Neural network controller

Neuralinputs

qd qd e e q q q1

d

Figure 5 A basic structure of the adaptive NN robot control

where 1198901 is the tracking error 1198902 is the velocity tracking error119878(119885) is the NN controller with being the weights matrixand 119878(119885) being the NN regressor vector and 1198701 and 1198702 arecontrol gains specified by the designer

From (10) we can see that the robot controller consists ofa PD-like controller and aNN controller In traditionalmodelbased controllers the dynamic model of the robot could beregarded as a feedforward to address the effect caused bythe robot motion In practice however 119872(119902) 119862(119902 119902) and119866(119902) may not be known Therefore the NNs are used toapproximate the unknown dynamics 119891 = 119872(119902) 119902 +119862( 119902 119902) 119902 +119866(119902) and to improve the performance of the system via theonline estimation To adapt theNNweights adaptive laws aredesigned as follows

119882 = Γ (119878119890 minus 120590) (11)

where Γ and 120590 are specified positive parametersThe last termof right-hand side of (11) is the sigma modification whichis used to enhance the convergence and robustness of theparameters adaptation

In [80] a NN based share control method was devel-oped to control a teleoperated robot with environmentaluncertainties In this work the RBFNN was constructed tocompensate for the unknown dynamics of the teleoperatedrobot Particularly a shared control strategy was developedinto the controller to achieve the automatic obstacle avoid-ance combining with the information of visual camera andthe robot body such that the obstacle could be successfullyavoided and the operator could focus more on the operatedtask rather than the environment to guarantee the stabilityand manipulation In addition error transformations wereintegrated into the adaptiveNN control to guarantee the tran-sient control performance It was shown that by using theNNtechnique the control performance in both kinematic leveland dynamic level of the teleoperated robot was enhancedIn [81] an extreme learning machine (ELM) based controlstrategy was proposed for uncertain robot manipulators toidentify both the elasticity and geometry of an object ThisELM was applied to deal with the unknown nonlinearity ofthe robot manipulator to enhance the control performanceParticularly by utilizing ELM the proposed controller couldguarantee that the robot dynamics follow a reference modelsuch that the desired set point and the feedforward forcecould be updated to estimate the geometry and stiffness ofthe object As a result the reference model could be exactlymatched with a limited number of iterations

In [82] the NN controller was also employed to control awheel inverted pendulum which has been decomposed intotwo subsystems a fully actuated second-order planar movingsubsystem and a passive first-order pendulum subsystemThen the RBFNN was employed to compensate for theuncertain dynamics of the two subsystems by using itspowerful learning ability such that the enhanced controlperformance could be realized by using the NN learning In[83] a global adaptive neural control was proposed for a classof robot manipulators with finite-time convergence learn-ing performance This control scheme employed a smoothswitching mechanism combining with a nominal neuralnetwork controller and a robust controller to ensure globaluniform ultimately bounded stability The optimal weightswere obtained by the finite-time estimation algorithm suchthat after the learning process the learning weights could bereused next time for repeated tasks The global NN controlmechanism has been further extended to the control of dualarm robot manipulator in [84] where knowledge of bothrobot manipulator and the grasping object is unavailable inadvance By integrating prescribed functions into the designof controller the transient performance of the dual armrobot control was regularly guaranteed The NN was alsoemployed to deal with synchronization problem of multiplerobot manipulators in [85] where the reference trajectoriesare only available for part of the team members By usingthe NN approximation controller the robot has shown bettercontrol performance with enhanced transient performanceand enhanced robustness A RBFNN was constructed tocompensate for the nonlinear terms of a five-bar manipulatorbased on an error transformation function [86] The NNcontrol was also applied in the robot teleoperation control[87 88] Moreover a NN approximation technique wasemployed to deal with the unknown dynamics kinematicsand actuator properties in the manipulator tracking control[89]

42 NN Based Robot Control with Input NonlinearitiesAnother challenge of the robot manipulator is that theinput nonlinearities such as friction dead-zone and actuatorsaturation may inevitably exist in the robot systems Theseinput nonlinearities may lead to larger tracking errors anddegeneration of the control performance Therefore a num-ber of works have been proposed to handle the nonlinearitiesby utilizing the neural network design A neural adaptive

Complexity 7

controller was designed to deal with the effect of inputsaturation of the robot manipulator in [90] as follows

120591 = minus1199111 + 119863119878119863 (119885119863) 1205721 + 119862119878119862 (119885119862) 1205721+ 119866119878119866 (119885119866) + 119870119901 (1199112 + 120585) + 119870119903 sgn (1199112) (12)

where 1199111 is the robot position tracking error 1199112 is the velocitytracking error and 1205721 is an auxiliary controller 119863 119862 and119866 are the NN weights 119878119863(119885119863) 119878119862(119885119862) and 119878119866(119885119866) arethe NN regressor vectors and 119870119901 and 119870119903 are control gainsspecified by the designer 120585 is an auxiliary system designed toreduce the effect of the saturation with 120585 defined as follows

120585

= minus119870120585120585 minus

100381610038161003816100381610038161199111198792Δ12059110038161003816100381610038161003816 + (12) Δ120591119879Δ120591100381710038171003817100381712058510038171003817100381710038172 120585 + Δ120591 10038171003817100381710038171205851003817100381710038171003817 ge 1205830 10038171003817100381710038171205851003817100381710038171003817 lt 120583

(13)

where Δ120591 is the torque error caused by saturation and119870120585 is asmall positive value To update the NNweights adaptive lawsare designed as follows

119882119863 = Γ119863119896 (11987811986311989612057211198902119896 minus 120590119863119896119863119896)119882119862 = Γ119862119896 (11987811986211989612057211198902119896 minus 120590119862119896119862119896)119882119866 = Γ119866119896 (11987811986611989612057211198902119896 minus 120590119866119896119866119896)

(14)

where Γ119863119896 Γ119862119896 and Γ119866119896 are specified positive parameters and120590119863119896 120590119862119896 and 120590119866119896 are positive parametersIn [91] an adaptive neural network controller was con-

structed to approximate the input dead-zone and the uncer-tain dynamics of the robotic manipulator while the outputconstraint was also considered in the feedback control In[92] the NN was applied for the estimation of the unknownmodel parameters of a marine surface vessel and in [93] thefull-state constraint of an n-link robotic manipulator wasachieved by using the NN control The NN controller wasalso constructed for flexible roboticmanipulators to deal withthe vibration suppression based on a lumped spring-massmodel [94] while in [95] two RBFNNs were constructed forflexible robot manipulators to compensate for the unknowndynamics and the dead-zone effect respectively

The NN has also been used in many important industrialfields such as autonomous underwater vehicles (AUVs) andhypersonic flight vehicle (HFV) In [96] the NN has beenconstructed to deal with the attitude of AUVs in the presenceof input dead-zone and uncertain model parameters In [97]the adaptive neural control was employed to deal with under-water vehicle control in discrete-time domain encounteredwith the unknown input nonlinearities external disturbanceand model uncertainties Then the reinforcement learningwas applied to address these uncertainties by using a criticNN and an action NN The hypersonic flight vehicle controlwas investigated in [98] where the aerodynamic uncertaintiesand unknown disturbances were addressed by a disturbance

observer based NN In [99] a neural learning control wasembedded in the HFV controller to achieve the globalstability via a switching mechanism and a robust controller

43 NN Based Human-Robot Interaction Control Recentlythere is a predominant tendency to employ the robots inthe human-surrounded environment such as householdservices or industrial applications where humans and robotsmay interact with each other directly Therefore interactioncontrol has become a promising research field and has beenwidely studied In [100] a learning method was developedsuch that the dynamics of a robot arm could follow atarget impedance model with only knowledge of the roboticstructure (see Figure 6)

TheNNwas further employed in robot control in interac-tionwith an environment [101] where impedance control wasachieved with the completely unknown robotic dynamicsIn [102] a learning method was developed such that therobot was able to adjust the impedance parameters when itinteracted with unknown environments In order to learnoptimal impedance parameters in the robot manipulatorcontrol an adaptive dynamic programming (ADP) methodwas employed when the robot interacted with unknowntime-varying environments where NNs were used for bothcritic and actor networks [103] The ADP was also employedfor coordination of multirobots [104] in which possibledisagreement between different manipulators was handledand dynamics of both robots and themanipulated object werenot required to be known

In this work the controller consists of two parts a criticnetwork which was used to approximate the cost functionand an actual NN which was designed to control the robotThe critic NN is designed as follows [104]

Υ (119905) = 119862119878119862 (119885119862) (15)

where 119885119862 = 120585 120585 = [119879119900 119911119879 119909119879119900 ]119879 with 119909119900 being the positionof the object and 119911 being the tracking error 119862 is the NNweight and 119878119862 is the regressor vector

The critic NN is used to approximate a cost function119888(119905) = 120585119879119876120585 + 119906119879119877119906 where 119906 denotes the control inputand 119876 and 119877 are positive definite matrix Since the controlobjective is to minimize the control effort the adaptation lawis designed as

119882119888 = minus120590119888 120597119864119888120597119888 = 120590119888 (119888 minus 119879119888 119878119862) 119878119862 (16)

where 120590119888 is the learning rate and 119864119888 = (12)(119888 minus 119879119888 119878119862)2On the other hand the actual NN control is designed to

control the robot as

119906 = 119879119886 119878119886 (119885119886) minus 119890 minus 1198702119890V (17)

where 119879119886 119878119886(119885119886) could learn the dynamics of the robotwith 119886 being the NN weight and 119878119886 being the regressorvector 119890 and 119890V are the position and velocity tracking errorsrespectively and 1198702 is the control gain

8 Complexity

Positon control Robotic arm Inverse

kinematics

Forward kinematics

Optimal critic learning

EnvironmentImpedance

model

xd

xrqr

x q

f

minus

+

Figure 6 Block of the learning impedence control

Since the control objective is to guarantee the estimationof both robot dynamics and the cost function Υ(119905) theadaptive law is selected as follows

119882119879119886119894 = minus120590119886 (119879119886119894119878119886 + 119896119903Υ) 119878119886 (18)

where 120590119886 and 119896119903 are positive constantsOn the other hand as a fundamental element of the next-

generation robots the human-robot collaboration (HRC) hasbeen widely studied by roboticists and NN is employed inHRCwith its powerful learning ability In [105] theNNswereemployed to estimate the human partnerrsquos motion intentionin human-robot collaboration such that the robot was able toactively follow its human partner To adjust the robotrsquos role tolead or to follow according to the humanrsquos intention gametheory was employed for fundamental analysis of human-robot interaction and an adaptation law was developed in[106] Policy iteration combining with NN was adopted toprovide a rigorous solution to the problem of the systemequilibrium in human-robot interaction [107]

44 NN Based Robot Cognitive Control According to thepredictive processing theory [108] the human brain is alwaysactively anticipating the incoming sensorimotor informationThis process exists because the living beings exhibit latenciesdue to neural processing delays and a limited bandwidthin their sensorimotor processing To compensate for sucha delay in human brain neural feedback signals (includinglateral and top-down connections)modulate the neural activ-ities via inhibitory or excitatory connections by influencingthe neuronal population coding of the bottom-up sensory-driven signals in the perception-action system Similarlyin robotic systems it is claimed that such a delay and alimited bandwidth also can be compensated by the predictivefunctions learnt by recurrent neural models Such a learningprocess can be done via only visual processing [109] or in theloop of perception and action [110]

Based on the hierarchical sensorimotor integration the-ory which advocates that action and perception are inter-twined by sharing the same representational basis [111] therepresentation on different levels of sensory perception doesnot explicitly represent actions instead there is an encodingof the possible future percept which is learnt from priorsensorimotor knowledge

In the Bayesian once this perception and action linkshave been established after learning these perception-action

associations in this architecture allow the following opera-tions

First these associations allow predicting the perceptualoutcome of given actions by means of the forward models(eg Bayesian model) It can be written as

119875 (119864 | 119860 119868) prop 119875 (119860 | 119864) 119875 (119864 | 119868) (19)

where E estimates the upcoming perception evidence givenan executed action A and other prior information you havealready known in 119868 The term 119875(119864 | 119860 119868) suggests that aprelearntmodel representing the possibility of amotor actionA will be executed given that a (possible) resulting sensoryevidence 119864 is perceived (backward computation)

Second these associations allow selecting an appropri-ate movement given an intended perceptual representationFrom the backward computations introduced in the followingequation a predictive sensorimotor integration occurs

119875 (119860 | 119864 119866) prop 119875 (119864 | 119860) 119875 (119860 | 119866) (20)

where A indicates a particular action selected given the(intended) sensory information 119864 and a goal G Here weassume that onersquos action is only determined by the currentsensory input and the goal

In terms of its hierarchical organization it also allows thisoperation with bidirectional information pathways a lowlevel perception representation can be expressed on a higherlevel with a more complex receptive field and vice versa119890low hArr 119890high This can be realized by the bidirectional deeparchitectures such as [112] Conceptually these operationscan be achieved by extracting statistical regularity shown inFigure 7

Since both perception and action processes can be seenas temporal sequences from the mathematical perspectivethe recurrent networks are Turing-Complete and have alearning capacity to learn time sequences with arbitrarylength [113] if properly trained Furthermore such recurrentconnections can be placed in a hierarchical way in whichthe prediction functions on different layers attempt to predictthe nonlinear time-series in different time-scales [114] Fromthis point the recurrent neural network with parametric biasunits (RNNPB) [115] andmultiple time-scale recurrent neuralnetworks (MTRNN) [116] were applied to predict sequencesby understanding them in various temporal levels

The difference of the temporal levels controls the prop-erties of the different levels of the presentation in the deep

Complexity 9

SMA

M1

Brainstem

Spine

Action

Common coding

Common coding

MST

V5MT

V3

V1

Cognitiveprocesses

Visual perception(dorsal)

Figure 7 The process of cognitive control

recurrent network For instance in the MTRNN network[112] the learning of each neuron follows the updating rule ofclassical firing rate models in which the activity of a neuronis determined by the average firing rate of all the connectedneurons Additionally the neuronal activity is also decayingover time following an updating rule of leaky integratormodel Assuming the i-th MTRNN neuron has the numberof N connections the current membrane potential status ofa neuron can be defined by both the previous activation andthe current synaptic inputs

119906119894119905+1 = (1 minus 1120591119894)119906119894119905 + 1120591119894 [sum119908119894119895119909119895119905] (21)

where 119908119894119895 represents the synaptic weight from the j-thneuron to the i-th neuron 119909119895119905 is the activity of j-th neuronat t-th time-step and 120591 is the time-scale parameter whichdetermines the decay rate of this neuron a larger 120591 meanstheir activities change slowly over time compared with thosewith a smaller time-scale parameter 120591

In [117] the concepts of predictive coding were discussedin detail where the learning generation and recognition ofactions can be conducted by means of the principle of pre-diction error minimization By using the predictive codingthe RNNPB and MTRNN are capable for both generatingown actions and recognizing the same actions performedby others Recently the study on neurorobotics experimentshas shown that the dynamic predictive coding scheme canbe used to address fluctuations in temporal patterns whentraining a recurrent neural network (RNN) model [118]This predictive coding scheme enables organisms to predictperceptual outcomes based on current intentions of actionsto the external environment and to forecast perceptualsequences corresponding to given intention states [118]

Based on this architecture two-layer RNN models wereutilized to extract visual information [119] and to under-stand intentions [120] or emotion status [121] in socialrobotics three-layer RNNmodels were used to integrate and

understand multimodal information for a humanoid iCubrobot [112 122] Moreover the predictive coding frameworkhas been extended to variational Bayes predictive codingMTRNN which can arbitrate between deterministic modeland probabilistic model by setting a metaparameter [123]Such extension could provide significant improvement indealing with noisy fluctuated sensory inputs which robots areexpected to experience in more real world setting In [124]a MTRNN was employed to control a humanoid robot andexperimental results have shown that by using only partialtraining data the control model can achieve generalizationby learning in a lower feature perception level

The hierarchical structure of RNN exhibits a greatlearning capacity to store multimodal information whichis beneficial for the robotic systems to understand and topredict in a complex environment As the future models andapplications the state-of-the-art deep learning techniques orthemotor actions of robotic systems can be further integratedinto this predictive architecture

5 Conclusion

In summary great achievements for control design of nonlin-ear system by means of neural networks have been gained inthe last two decades Despite the impossibility in identifyingor listing all the related contributions in this short reviewefforts have been made to summarize the recent progressin the area of NN control and its particular applications inthe robot learning control the robot interaction control andthe robot recognition control In this paper we have shownthat significant progress of NN has been made in control ofthe nonlinear systems in solving the optimization problemin approximating the system dynamics in dealing with theinput nonlinearities in human-robot interaction and in thepattern recognition All these developments accompany notonly the development of techniques in control and advancedmanufactures but also theatrical progress in constructingand developing the neural networks Although huge efforts

10 Complexity

have been made to embed the NN in practical control sys-tems there is still a large gap between the theory and practiceTo improve the feasibility and usability the evolutionarycomputing theory has been proposed to train the NNs It canautomatically find a near-optimal NN architecture and allowa NN to adapt its learning rule to its environment Howeverthe complex and long training process of the evolutionaryalgorithms deters their practical applications More effortsneed to be made to evolve the NN architecture and NNlearning technique in the control design On the otherhand in human brain the neural activities are modulatedvia inhibitory or excitatory connections by influencing theneuronal population coding of the bottom-up sensory-drivensignals in the perception-action system In this sense howto integrate the sensor-motor information into the networkto make NNs more feasible to adapt to the environment andto resemble the capacity of the human brain deserves furtherinvestigations

In conclusion a brief review on neural networks for thecomplex nonlinear systems is provided with adaptive neuralcontrol NN based dynamic programming evolution com-puting and their practical applications in the robotic fieldsWe believe this area may promote increasing investigationsin both theories and applications And emerging topics likedeep learning [125ndash128] big data [129ndash131] and cloud com-puting may be incorporated into the neural network controlfor complex systems for example deep neural networkscould be used to process massive amounts of unsuperviseddata in complex scenarios neural networks can be helpfulin reducing the data dimensionality and the optimization ofNN training may be employed to enhance the learning andadaptation performance of robots

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

Thisworkwas partially supported by theNational Nature Sci-ence Foundation (NSFC) under Grant 61473120 GuangdongProvincial Natural Science Foundation 2014A030313266International Science and Technology Collaboration Grant2015A050502017 Science andTechnology Planning Project ofGuangzhou 201607010006 State Key Laboratory of Roboticsand System (HIT) Grant SKLRS-2017-KF-13 and the Funda-mental Research Funds for the Central Universities

References

[1] F Gruau D Whitley and L Pyeatt ldquoA comparison betweencellular encoding and direct encoding for genetic neural net-worksrdquo inProceedings of the 1st annual conference on genetic pro-gramming pp 81ndash89 1996

[2] H de Garis ldquoAn artificial brain ATRrsquos CAM-Brain Project aimsto buildevolve an artificial brain with a million neural netmodules inside a trillion cell Cellular Automata Machinerdquo NewGeneration Computing vol 12 no 2 pp 215ndash221 1994

[3] W S McCulloch and W Pitts ldquoA logical calculus of the ideasimmanent in nervous activityrdquoBulletin ofMathematical Biologyvol 5 no 4 pp 115ndash133 1943

[4] C Yang S S Ge C Xiang T Chai and T H Lee ldquoOutput feed-back NN control for two classes of discrete-time systems withunknown control directions in a unified approachrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 19 no 11pp 1873ndash1886 2008

[5] R Kumar R K Aggarwal and J D Sharma ldquoEnergy analysis ofa building using artificial neural network A reviewrdquo Energy andBuildings vol 65 pp 352ndash358 2013

[6] C Alippi C De Russis and V Piuri ldquoA neural-network basedcontrol solution to air-fuel ratio control for automotive fuel-injection systemsrdquo IEEE Transactions on Systems Man andCybernetics Part C Applications and Reviews vol 33 no 2 pp259ndash268 2003

[7] E M Azoff Neural network time series forecasting of financialmarkets John Wiley amp Sons Inc 1994

[8] I Fister P N Suganthan J Fister et al ldquoArtificial neural net-work regression as a local search heuristic for ensemble strate-gies in differential evolutionrdquo Nonlinear Dynamics vol 84 no2 pp 895ndash914 2016

[9] Y Zhang S Li and H Guo ldquoA type of biased consensus-based distributed neural network for path planningrdquoNonlinearDynamics vol 89 no 3 pp 1803ndash1815 2017

[10] X Shi Z Wang and L Han ldquoFinite-time stochastic synchro-nization of time-delay neural networks with noise disturbancerdquoNonlinear Dynamics vol 88 no 4 pp 2747ndash2755 2017

[11] P He and Y Li ldquoHinfin synchronization of coupled reaction-diffusion neural networks with mixed delaysrdquo Complexity vol21 no S2 pp 42ndash53 2016

[12] Z Tu J Cao A Alsaedi F E Alsaadi and T Hayat ldquoGlobalLagrange stability of complex-valued neural networks of neutraltype with time-varying delaysrdquo Complexity vol 21 no S2 pp438ndash450 2016

[13] C Wang S Guo and Y Xu ldquoFormation of autapse connectedto neuron and its biological functionrdquo Complexity vol 2017Article ID 5436737 9 pages 2017

[14] J D J Rubio I Elias D R Cruz and J Pacheco ldquoUniform stableradial basis function neural network for the prediction in twomechatronic processesrdquo Neurocomputing vol 227 pp 122ndash1302017

[15] J d Rubio ldquoUSNFIS Uniform stable neuro fuzzy inferencesystemrdquo Neurocomputing vol 262 pp 57ndash66 2017

[16] Q Liu J Yin V C M Leung J-H Zhai Z Cai and J LinldquoApplying a new localized generalization error model to designneural networks trained with extreme learning machinerdquo Neu-ral Computing and Applications pp 1ndash8 2014

[17] R J de Jesus ldquoInterpolation neural network model of a manu-factured wind turbinerdquoNeural Computing and Applications pp1ndash12 2016

[18] C Mu and D Wang ldquoNeural-network-based adaptive guaran-teed cost control of nonlinear dynamical systems with matcheduncertaintiesrdquo Neurocomputing vol 245 pp 46ndash54 2017

[19] Z Lin DMa J Meng and L Chen ldquoRelative ordering learningin spiking neural network for pattern recognitionrdquo Neuro-computing 2017

[20] J Yu J Sang and X Gao ldquoMachine learning and signal pro-cessing for big multimedia analysisrdquo Neurocomputing vol 257pp 1ndash4 2017

Complexity 11

[21] T Zhang S S Ge and C C Hang ldquoAdaptive neural networkcontrol for strict-feedback nonlinear systems using backstep-ping designrdquo Automatica vol 36 no 12 pp 1835ndash1846 2000

[22] S S Ge and T Zhang ldquoNeural-network control of nonaffinenonlinear system with zero dynamics by state and outputfeedbackrdquo IEEE Transactions on Neural Networks and LearningSystems vol 14 no 4 pp 900ndash918 2003

[23] S S Ge C Yang and T H Lee ldquoAdaptive predictive controlusing neural network for a class of pure-feedback systems in dis-crete timerdquo IEEE Transactions on Neural Networks and LearningSystems vol 19 no 9 pp 1599ndash1614 2008

[24] F W Lewis S Jagannathan and A Yesildirak Neural networkcontrol of robotmanipulators and non-linear systems CRCPress1998

[25] S Jagannathan and F L Lewis ldquoIdentification of nonlineardynamical systems using multilayered neural networksrdquo Auto-matica vol 32 no 12 pp 1707ndash1712 1996

[26] D Vrabie and F Lewis ldquoNeural network approach tocontinuous-time direct adaptive optimal control for partiallyunknown nonlinear systemsrdquo Neural Networks vol 22 no 3pp 237ndash246 2009

[27] C Yang S S Ge and T H Lee ldquoOutput feedback adaptive con-trol of a class of nonlinear discrete-time systems with unknowncontrol directionsrdquo Automatica vol 45 no 1 pp 270ndash2762009

[28] C Yang Z Li and J Li ldquoTrajectory planning and optimizedadaptive control for a class of wheeled inverted pendulumvehicle modelsrdquo IEEE Transactions on Cybernetics vol 43 no 1pp 24ndash36 2013

[29] Y Jiang C Yang and H Ma ldquoA review of fuzzy logic andneural network based intelligent control design for discrete-time systemsrdquo Discrete Dynamics in Nature and Society ArticleID 7217364 Art ID 7217364 11 pages 2016

[30] Y Jiang C Yang S-L Dai and B Ren ldquoDeterministic learningenhanced neutral network control of unmanned helicopterrdquoInternational Journal of Advanced Robotic Systems vol 13 no6 pp 1ndash12 2016

[31] Y Jiang Z Liu C Chen and Y Zhang ldquoAdaptive robust fuzzycontrol for dual arm robot with unknown input deadzonenonlinearityrdquo Nonlinear Dynamics vol 81 no 3 pp 1301ndash13142015

[32] httpsstemcellthailandorgneurons-definition-function-neurotransmitters

[33] MDefoort T Floquet A Kokosy andW Perruquetti ldquoSliding-mode formation control for cooperative autonomous mobilerobotsrdquo IEEE Transactions on Industrial Electronics vol 55 no11 pp 3944ndash3953 2008

[34] X Liu C Yang Z Chen MWang and C Su ldquoNeuro-adaptiveobserver based control of flexible joint robotrdquoNeurocomputing2017

[35] F Hamerlain T Floquet and W Perruquetti ldquoExperimentaltests of a slidingmode controller for trajectory tracking of a car-like mobile robotrdquo Robotica vol 32 no 1 pp 63ndash76 2014

[36] R J de Jesus ldquoDiscrete time control based in neural networksfor pendulumsrdquo Applied Soft Computing 2017

[37] Y Pan M J Er T Sun B Xu and H Yu ldquoAdaptive fuzzy PDcontrol with stable Hinfin tracking guaranteerdquo Neurocomputingvol 237 pp 71ndash78 2017

[38] R J de Jesus ldquoAdaptive least square control in discrete time ofrobotic armsrdquo Soft Computing vol 19 no 12 pp 3665ndash36762015

[39] S Commuri S Jagannathan and F L Lewis ldquoCMAC neuralnetwork control of robot manipulatorsrdquo Journal of RoboticSystems vol 14 no 6 pp 465ndash482 1997

[40] J S Albus ldquoTheoretical and experimental aspects of a cerebellarmodelrdquoDevelopmental Disabilities Research Reviews vol 17 pp93ndash101 1972

[41] B Yang R Bao and H Han ldquoRobust hybrid control based onPD and novel CMAC with improved architecture and learningscheme for electric load simulatorrdquo IEEE Transactions onIndustrial Electronics vol 61 no 10 pp 5271ndash5279 2014

[42] S Jagannathan and F L Lewis ldquoMultilayer discrete-timeneural-net controller with guaranteed performancerdquo IEEETransactions on Neural Networks and Learning Systems vol 7no 1 pp 107ndash130 1996

[43] S S Ge and J Wang ldquoRobust adaptive neural control for a classof perturbed strict feedback nonlinear systemsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no6 pp 1409ndash1419 2002

[44] Y H Kim F L Lewis and C T Abdallah ldquoA dynamic recurrentneural-network-based adaptive observer for a class of nonlinearsystemsrdquo Automatica vol 33 no 8 pp 1539ndash1543 1997

[45] J-Q Huang and F L Lewis ldquoNeural-network predictive controlfor nonlinear dynamic systems with time-delayrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 14 no 2pp 377ndash389 2003

[46] S S Ge F Hong and T H Lee ldquoAdaptive neural control ofnonlinear time-delay systems with unknown virtual controlcoefficientsrdquo IEEE Transactions on Systems Man and Cybernet-ics Part B Cybernetics vol 34 no 1 pp 499ndash516 2004

[47] J Na X M Ren and H Huang ldquoTime-delay positive feedbackcontrol for nonlinear time-delay systems with neural networkcompensationrdquo Zidonghua XuebaoActa Automatica Sinica vol34 no 9 pp 1196ndash1202 2008

[48] J Na X Ren C Shang and Y Guo ldquoAdaptive neural networkpredictive control for nonlinear pure feedback systems withinput delayrdquo Journal of Process Control vol 22 no 1 pp 194ndash206 2012

[49] R R Selmic and F L Lewis ldquoNeural-network approximationof piecewise continuous functions application to friction com-pensationrdquo IEEE Transactions on Neural Networks and LearningSystems vol 13 no 3 pp 745ndash751 2002

[50] H Zhang and F L Lewis ldquoAdaptive cooperative trackingcontrol of higher-order nonlinear systems with unknowndynamicsrdquo Automatica vol 48 no 7 pp 1432ndash1439 2012

[51] J Na X Ren and D Zheng ldquoAdaptive control for nonlinearpure-feedback systems with high-order sliding mode observerrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 24 no 3 pp 370ndash382 2013

[52] J Na Q Chen X Ren and Y Guo ldquoAdaptive prescribedperformancemotion control of servomechanisms with frictioncompensationrdquo IEEE Transactions on Industrial Electronics vol61 no 1 pp 486ndash494 2014

[53] J Na X Ren G Herrmann and Z Qiao ldquoAdaptive neuraldynamic surface control for servo systems with unknown dead-zonerdquoControl Engineering Practice vol 19 no 11 pp 1328ndash13432011

[54] G Li J Na D P Stoten and X Ren ldquoAdaptive neural networkfeedforward control for dynamically substructured systemsrdquoIEEE Transactions on Control Systems Technology vol 22 no3 pp 944ndash954 2014

Complexity 7

controller was designed to deal with the effect of inputsaturation of the robot manipulator in [90] as follows

120591 = minus1199111 + 119863119878119863 (119885119863) 1205721 + 119862119878119862 (119885119862) 1205721+ 119866119878119866 (119885119866) + 119870119901 (1199112 + 120585) + 119870119903 sgn (1199112) (12)

where 1199111 is the robot position tracking error 1199112 is the velocitytracking error and 1205721 is an auxiliary controller 119863 119862 and119866 are the NN weights 119878119863(119885119863) 119878119862(119885119862) and 119878119866(119885119866) arethe NN regressor vectors and 119870119901 and 119870119903 are control gainsspecified by the designer 120585 is an auxiliary system designed toreduce the effect of the saturation with 120585 defined as follows

120585

= minus119870120585120585 minus

100381610038161003816100381610038161199111198792Δ12059110038161003816100381610038161003816 + (12) Δ120591119879Δ120591100381710038171003817100381712058510038171003817100381710038172 120585 + Δ120591 10038171003817100381710038171205851003817100381710038171003817 ge 1205830 10038171003817100381710038171205851003817100381710038171003817 lt 120583

(13)

where Δ120591 is the torque error caused by saturation and119870120585 is asmall positive value To update the NNweights adaptive lawsare designed as follows

119882119863 = Γ119863119896 (11987811986311989612057211198902119896 minus 120590119863119896119863119896)119882119862 = Γ119862119896 (11987811986211989612057211198902119896 minus 120590119862119896119862119896)119882119866 = Γ119866119896 (11987811986611989612057211198902119896 minus 120590119866119896119866119896)

(14)

where Γ119863119896 Γ119862119896 and Γ119866119896 are specified positive parameters and120590119863119896 120590119862119896 and 120590119866119896 are positive parametersIn [91] an adaptive neural network controller was con-

structed to approximate the input dead-zone and the uncer-tain dynamics of the robotic manipulator while the outputconstraint was also considered in the feedback control In[92] the NN was applied for the estimation of the unknownmodel parameters of a marine surface vessel and in [93] thefull-state constraint of an n-link robotic manipulator wasachieved by using the NN control The NN controller wasalso constructed for flexible roboticmanipulators to deal withthe vibration suppression based on a lumped spring-massmodel [94] while in [95] two RBFNNs were constructed forflexible robot manipulators to compensate for the unknowndynamics and the dead-zone effect respectively

The NN has also been used in many important industrialfields such as autonomous underwater vehicles (AUVs) andhypersonic flight vehicle (HFV) In [96] the NN has beenconstructed to deal with the attitude of AUVs in the presenceof input dead-zone and uncertain model parameters In [97]the adaptive neural control was employed to deal with under-water vehicle control in discrete-time domain encounteredwith the unknown input nonlinearities external disturbanceand model uncertainties Then the reinforcement learningwas applied to address these uncertainties by using a criticNN and an action NN The hypersonic flight vehicle controlwas investigated in [98] where the aerodynamic uncertaintiesand unknown disturbances were addressed by a disturbance

observer based NN In [99] a neural learning control wasembedded in the HFV controller to achieve the globalstability via a switching mechanism and a robust controller

43 NN Based Human-Robot Interaction Control Recentlythere is a predominant tendency to employ the robots inthe human-surrounded environment such as householdservices or industrial applications where humans and robotsmay interact with each other directly Therefore interactioncontrol has become a promising research field and has beenwidely studied In [100] a learning method was developedsuch that the dynamics of a robot arm could follow atarget impedance model with only knowledge of the roboticstructure (see Figure 6)

TheNNwas further employed in robot control in interac-tionwith an environment [101] where impedance control wasachieved with the completely unknown robotic dynamicsIn [102] a learning method was developed such that therobot was able to adjust the impedance parameters when itinteracted with unknown environments In order to learnoptimal impedance parameters in the robot manipulatorcontrol an adaptive dynamic programming (ADP) methodwas employed when the robot interacted with unknowntime-varying environments where NNs were used for bothcritic and actor networks [103] The ADP was also employedfor coordination of multirobots [104] in which possibledisagreement between different manipulators was handledand dynamics of both robots and themanipulated object werenot required to be known

In this work the controller consists of two parts a criticnetwork which was used to approximate the cost functionand an actual NN which was designed to control the robotThe critic NN is designed as follows [104]

Υ (119905) = 119862119878119862 (119885119862) (15)

where 119885119862 = 120585 120585 = [119879119900 119911119879 119909119879119900 ]119879 with 119909119900 being the positionof the object and 119911 being the tracking error 119862 is the NNweight and 119878119862 is the regressor vector

The critic NN is used to approximate a cost function119888(119905) = 120585119879119876120585 + 119906119879119877119906 where 119906 denotes the control inputand 119876 and 119877 are positive definite matrix Since the controlobjective is to minimize the control effort the adaptation lawis designed as

119882119888 = minus120590119888 120597119864119888120597119888 = 120590119888 (119888 minus 119879119888 119878119862) 119878119862 (16)

where 120590119888 is the learning rate and 119864119888 = (12)(119888 minus 119879119888 119878119862)2On the other hand the actual NN control is designed to

control the robot as

119906 = 119879119886 119878119886 (119885119886) minus 119890 minus 1198702119890V (17)

where 119879119886 119878119886(119885119886) could learn the dynamics of the robotwith 119886 being the NN weight and 119878119886 being the regressorvector 119890 and 119890V are the position and velocity tracking errorsrespectively and 1198702 is the control gain

8 Complexity

Positon control Robotic arm Inverse

kinematics

Forward kinematics

Optimal critic learning

EnvironmentImpedance

model

xd

xrqr

x q

f

minus

+

Figure 6 Block of the learning impedence control

Since the control objective is to guarantee the estimationof both robot dynamics and the cost function Υ(119905) theadaptive law is selected as follows

119882119879119886119894 = minus120590119886 (119879119886119894119878119886 + 119896119903Υ) 119878119886 (18)

where 120590119886 and 119896119903 are positive constantsOn the other hand as a fundamental element of the next-

generation robots the human-robot collaboration (HRC) hasbeen widely studied by roboticists and NN is employed inHRCwith its powerful learning ability In [105] theNNswereemployed to estimate the human partnerrsquos motion intentionin human-robot collaboration such that the robot was able toactively follow its human partner To adjust the robotrsquos role tolead or to follow according to the humanrsquos intention gametheory was employed for fundamental analysis of human-robot interaction and an adaptation law was developed in[106] Policy iteration combining with NN was adopted toprovide a rigorous solution to the problem of the systemequilibrium in human-robot interaction [107]

44 NN Based Robot Cognitive Control According to thepredictive processing theory [108] the human brain is alwaysactively anticipating the incoming sensorimotor informationThis process exists because the living beings exhibit latenciesdue to neural processing delays and a limited bandwidthin their sensorimotor processing To compensate for sucha delay in human brain neural feedback signals (includinglateral and top-down connections)modulate the neural activ-ities via inhibitory or excitatory connections by influencingthe neuronal population coding of the bottom-up sensory-driven signals in the perception-action system Similarlyin robotic systems it is claimed that such a delay and alimited bandwidth also can be compensated by the predictivefunctions learnt by recurrent neural models Such a learningprocess can be done via only visual processing [109] or in theloop of perception and action [110]

Based on the hierarchical sensorimotor integration the-ory which advocates that action and perception are inter-twined by sharing the same representational basis [111] therepresentation on different levels of sensory perception doesnot explicitly represent actions instead there is an encodingof the possible future percept which is learnt from priorsensorimotor knowledge

In the Bayesian once this perception and action linkshave been established after learning these perception-action

associations in this architecture allow the following opera-tions

First these associations allow predicting the perceptualoutcome of given actions by means of the forward models(eg Bayesian model) It can be written as

119875 (119864 | 119860 119868) prop 119875 (119860 | 119864) 119875 (119864 | 119868) (19)

where E estimates the upcoming perception evidence givenan executed action A and other prior information you havealready known in 119868 The term 119875(119864 | 119860 119868) suggests that aprelearntmodel representing the possibility of amotor actionA will be executed given that a (possible) resulting sensoryevidence 119864 is perceived (backward computation)

Second these associations allow selecting an appropri-ate movement given an intended perceptual representationFrom the backward computations introduced in the followingequation a predictive sensorimotor integration occurs

119875 (119860 | 119864 119866) prop 119875 (119864 | 119860) 119875 (119860 | 119866) (20)

where A indicates a particular action selected given the(intended) sensory information 119864 and a goal G Here weassume that onersquos action is only determined by the currentsensory input and the goal

In terms of its hierarchical organization it also allows thisoperation with bidirectional information pathways a lowlevel perception representation can be expressed on a higherlevel with a more complex receptive field and vice versa119890low hArr 119890high This can be realized by the bidirectional deeparchitectures such as [112] Conceptually these operationscan be achieved by extracting statistical regularity shown inFigure 7

Since both perception and action processes can be seenas temporal sequences from the mathematical perspectivethe recurrent networks are Turing-Complete and have alearning capacity to learn time sequences with arbitrarylength [113] if properly trained Furthermore such recurrentconnections can be placed in a hierarchical way in whichthe prediction functions on different layers attempt to predictthe nonlinear time-series in different time-scales [114] Fromthis point the recurrent neural network with parametric biasunits (RNNPB) [115] andmultiple time-scale recurrent neuralnetworks (MTRNN) [116] were applied to predict sequencesby understanding them in various temporal levels

The difference of the temporal levels controls the prop-erties of the different levels of the presentation in the deep

Complexity 9

SMA

M1

Brainstem

Spine

Action

Common coding

Common coding

MST

V5MT

V3

V1

Cognitiveprocesses

Visual perception(dorsal)

Figure 7 The process of cognitive control

recurrent network For instance in the MTRNN network[112] the learning of each neuron follows the updating rule ofclassical firing rate models in which the activity of a neuronis determined by the average firing rate of all the connectedneurons Additionally the neuronal activity is also decayingover time following an updating rule of leaky integratormodel Assuming the i-th MTRNN neuron has the numberof N connections the current membrane potential status ofa neuron can be defined by both the previous activation andthe current synaptic inputs

119906119894119905+1 = (1 minus 1120591119894)119906119894119905 + 1120591119894 [sum119908119894119895119909119895119905] (21)

where 119908119894119895 represents the synaptic weight from the j-thneuron to the i-th neuron 119909119895119905 is the activity of j-th neuronat t-th time-step and 120591 is the time-scale parameter whichdetermines the decay rate of this neuron a larger 120591 meanstheir activities change slowly over time compared with thosewith a smaller time-scale parameter 120591

In [117] the concepts of predictive coding were discussedin detail where the learning generation and recognition ofactions can be conducted by means of the principle of pre-diction error minimization By using the predictive codingthe RNNPB and MTRNN are capable for both generatingown actions and recognizing the same actions performedby others Recently the study on neurorobotics experimentshas shown that the dynamic predictive coding scheme canbe used to address fluctuations in temporal patterns whentraining a recurrent neural network (RNN) model [118]This predictive coding scheme enables organisms to predictperceptual outcomes based on current intentions of actionsto the external environment and to forecast perceptualsequences corresponding to given intention states [118]

Based on this architecture two-layer RNN models wereutilized to extract visual information [119] and to under-stand intentions [120] or emotion status [121] in socialrobotics three-layer RNNmodels were used to integrate and

understand multimodal information for a humanoid iCubrobot [112 122] Moreover the predictive coding frameworkhas been extended to variational Bayes predictive codingMTRNN which can arbitrate between deterministic modeland probabilistic model by setting a metaparameter [123]Such extension could provide significant improvement indealing with noisy fluctuated sensory inputs which robots areexpected to experience in more real world setting In [124]a MTRNN was employed to control a humanoid robot andexperimental results have shown that by using only partialtraining data the control model can achieve generalizationby learning in a lower feature perception level

The hierarchical structure of RNN exhibits a greatlearning capacity to store multimodal information whichis beneficial for the robotic systems to understand and topredict in a complex environment As the future models andapplications the state-of-the-art deep learning techniques orthemotor actions of robotic systems can be further integratedinto this predictive architecture

5 Conclusion

In summary great achievements for control design of nonlin-ear system by means of neural networks have been gained inthe last two decades Despite the impossibility in identifyingor listing all the related contributions in this short reviewefforts have been made to summarize the recent progressin the area of NN control and its particular applications inthe robot learning control the robot interaction control andthe robot recognition control In this paper we have shownthat significant progress of NN has been made in control ofthe nonlinear systems in solving the optimization problemin approximating the system dynamics in dealing with theinput nonlinearities in human-robot interaction and in thepattern recognition All these developments accompany notonly the development of techniques in control and advancedmanufactures but also theatrical progress in constructingand developing the neural networks Although huge efforts

10 Complexity

have been made to embed the NN in practical control sys-tems there is still a large gap between the theory and practiceTo improve the feasibility and usability the evolutionarycomputing theory has been proposed to train the NNs It canautomatically find a near-optimal NN architecture and allowa NN to adapt its learning rule to its environment Howeverthe complex and long training process of the evolutionaryalgorithms deters their practical applications More effortsneed to be made to evolve the NN architecture and NNlearning technique in the control design On the otherhand in human brain the neural activities are modulatedvia inhibitory or excitatory connections by influencing theneuronal population coding of the bottom-up sensory-drivensignals in the perception-action system In this sense howto integrate the sensor-motor information into the networkto make NNs more feasible to adapt to the environment andto resemble the capacity of the human brain deserves furtherinvestigations

In conclusion a brief review on neural networks for thecomplex nonlinear systems is provided with adaptive neuralcontrol NN based dynamic programming evolution com-puting and their practical applications in the robotic fieldsWe believe this area may promote increasing investigationsin both theories and applications And emerging topics likedeep learning [125ndash128] big data [129ndash131] and cloud com-puting may be incorporated into the neural network controlfor complex systems for example deep neural networkscould be used to process massive amounts of unsuperviseddata in complex scenarios neural networks can be helpfulin reducing the data dimensionality and the optimization ofNN training may be employed to enhance the learning andadaptation performance of robots

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

Thisworkwas partially supported by theNational Nature Sci-ence Foundation (NSFC) under Grant 61473120 GuangdongProvincial Natural Science Foundation 2014A030313266International Science and Technology Collaboration Grant2015A050502017 Science andTechnology Planning Project ofGuangzhou 201607010006 State Key Laboratory of Roboticsand System (HIT) Grant SKLRS-2017-KF-13 and the Funda-mental Research Funds for the Central Universities

References

[1] F Gruau D Whitley and L Pyeatt ldquoA comparison betweencellular encoding and direct encoding for genetic neural net-worksrdquo inProceedings of the 1st annual conference on genetic pro-gramming pp 81ndash89 1996

[2] H de Garis ldquoAn artificial brain ATRrsquos CAM-Brain Project aimsto buildevolve an artificial brain with a million neural netmodules inside a trillion cell Cellular Automata Machinerdquo NewGeneration Computing vol 12 no 2 pp 215ndash221 1994

[3] W S McCulloch and W Pitts ldquoA logical calculus of the ideasimmanent in nervous activityrdquoBulletin ofMathematical Biologyvol 5 no 4 pp 115ndash133 1943

[4] C Yang S S Ge C Xiang T Chai and T H Lee ldquoOutput feed-back NN control for two classes of discrete-time systems withunknown control directions in a unified approachrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 19 no 11pp 1873ndash1886 2008

[5] R Kumar R K Aggarwal and J D Sharma ldquoEnergy analysis ofa building using artificial neural network A reviewrdquo Energy andBuildings vol 65 pp 352ndash358 2013

[6] C Alippi C De Russis and V Piuri ldquoA neural-network basedcontrol solution to air-fuel ratio control for automotive fuel-injection systemsrdquo IEEE Transactions on Systems Man andCybernetics Part C Applications and Reviews vol 33 no 2 pp259ndash268 2003

[7] E M Azoff Neural network time series forecasting of financialmarkets John Wiley amp Sons Inc 1994

[8] I Fister P N Suganthan J Fister et al ldquoArtificial neural net-work regression as a local search heuristic for ensemble strate-gies in differential evolutionrdquo Nonlinear Dynamics vol 84 no2 pp 895ndash914 2016

[9] Y Zhang S Li and H Guo ldquoA type of biased consensus-based distributed neural network for path planningrdquoNonlinearDynamics vol 89 no 3 pp 1803ndash1815 2017

[10] X Shi Z Wang and L Han ldquoFinite-time stochastic synchro-nization of time-delay neural networks with noise disturbancerdquoNonlinear Dynamics vol 88 no 4 pp 2747ndash2755 2017

[11] P He and Y Li ldquoHinfin synchronization of coupled reaction-diffusion neural networks with mixed delaysrdquo Complexity vol21 no S2 pp 42ndash53 2016

[12] Z Tu J Cao A Alsaedi F E Alsaadi and T Hayat ldquoGlobalLagrange stability of complex-valued neural networks of neutraltype with time-varying delaysrdquo Complexity vol 21 no S2 pp438ndash450 2016

[13] C Wang S Guo and Y Xu ldquoFormation of autapse connectedto neuron and its biological functionrdquo Complexity vol 2017Article ID 5436737 9 pages 2017

[14] J D J Rubio I Elias D R Cruz and J Pacheco ldquoUniform stableradial basis function neural network for the prediction in twomechatronic processesrdquo Neurocomputing vol 227 pp 122ndash1302017

[15] J d Rubio ldquoUSNFIS Uniform stable neuro fuzzy inferencesystemrdquo Neurocomputing vol 262 pp 57ndash66 2017

[16] Q Liu J Yin V C M Leung J-H Zhai Z Cai and J LinldquoApplying a new localized generalization error model to designneural networks trained with extreme learning machinerdquo Neu-ral Computing and Applications pp 1ndash8 2014

[17] R J de Jesus ldquoInterpolation neural network model of a manu-factured wind turbinerdquoNeural Computing and Applications pp1ndash12 2016

[18] C Mu and D Wang ldquoNeural-network-based adaptive guaran-teed cost control of nonlinear dynamical systems with matcheduncertaintiesrdquo Neurocomputing vol 245 pp 46ndash54 2017

[19] Z Lin DMa J Meng and L Chen ldquoRelative ordering learningin spiking neural network for pattern recognitionrdquo Neuro-computing 2017

[20] J Yu J Sang and X Gao ldquoMachine learning and signal pro-cessing for big multimedia analysisrdquo Neurocomputing vol 257pp 1ndash4 2017

Complexity 11

[21] T Zhang S S Ge and C C Hang ldquoAdaptive neural networkcontrol for strict-feedback nonlinear systems using backstep-ping designrdquo Automatica vol 36 no 12 pp 1835ndash1846 2000

[22] S S Ge and T Zhang ldquoNeural-network control of nonaffinenonlinear system with zero dynamics by state and outputfeedbackrdquo IEEE Transactions on Neural Networks and LearningSystems vol 14 no 4 pp 900ndash918 2003

[23] S S Ge C Yang and T H Lee ldquoAdaptive predictive controlusing neural network for a class of pure-feedback systems in dis-crete timerdquo IEEE Transactions on Neural Networks and LearningSystems vol 19 no 9 pp 1599ndash1614 2008

[24] F W Lewis S Jagannathan and A Yesildirak Neural networkcontrol of robotmanipulators and non-linear systems CRCPress1998

[25] S Jagannathan and F L Lewis ldquoIdentification of nonlineardynamical systems using multilayered neural networksrdquo Auto-matica vol 32 no 12 pp 1707ndash1712 1996

[26] D Vrabie and F Lewis ldquoNeural network approach tocontinuous-time direct adaptive optimal control for partiallyunknown nonlinear systemsrdquo Neural Networks vol 22 no 3pp 237ndash246 2009

[27] C Yang S S Ge and T H Lee ldquoOutput feedback adaptive con-trol of a class of nonlinear discrete-time systems with unknowncontrol directionsrdquo Automatica vol 45 no 1 pp 270ndash2762009

[28] C Yang Z Li and J Li ldquoTrajectory planning and optimizedadaptive control for a class of wheeled inverted pendulumvehicle modelsrdquo IEEE Transactions on Cybernetics vol 43 no 1pp 24ndash36 2013

[29] Y Jiang C Yang and H Ma ldquoA review of fuzzy logic andneural network based intelligent control design for discrete-time systemsrdquo Discrete Dynamics in Nature and Society ArticleID 7217364 Art ID 7217364 11 pages 2016

[30] Y Jiang C Yang S-L Dai and B Ren ldquoDeterministic learningenhanced neutral network control of unmanned helicopterrdquoInternational Journal of Advanced Robotic Systems vol 13 no6 pp 1ndash12 2016

[31] Y Jiang Z Liu C Chen and Y Zhang ldquoAdaptive robust fuzzycontrol for dual arm robot with unknown input deadzonenonlinearityrdquo Nonlinear Dynamics vol 81 no 3 pp 1301ndash13142015

[32] httpsstemcellthailandorgneurons-definition-function-neurotransmitters

[33] MDefoort T Floquet A Kokosy andW Perruquetti ldquoSliding-mode formation control for cooperative autonomous mobilerobotsrdquo IEEE Transactions on Industrial Electronics vol 55 no11 pp 3944ndash3953 2008

[34] X Liu C Yang Z Chen MWang and C Su ldquoNeuro-adaptiveobserver based control of flexible joint robotrdquoNeurocomputing2017

[35] F Hamerlain T Floquet and W Perruquetti ldquoExperimentaltests of a slidingmode controller for trajectory tracking of a car-like mobile robotrdquo Robotica vol 32 no 1 pp 63ndash76 2014

[36] R J de Jesus ldquoDiscrete time control based in neural networksfor pendulumsrdquo Applied Soft Computing 2017

[37] Y Pan M J Er T Sun B Xu and H Yu ldquoAdaptive fuzzy PDcontrol with stable Hinfin tracking guaranteerdquo Neurocomputingvol 237 pp 71ndash78 2017

[38] R J de Jesus ldquoAdaptive least square control in discrete time ofrobotic armsrdquo Soft Computing vol 19 no 12 pp 3665ndash36762015

[39] S Commuri S Jagannathan and F L Lewis ldquoCMAC neuralnetwork control of robot manipulatorsrdquo Journal of RoboticSystems vol 14 no 6 pp 465ndash482 1997

[40] J S Albus ldquoTheoretical and experimental aspects of a cerebellarmodelrdquoDevelopmental Disabilities Research Reviews vol 17 pp93ndash101 1972

[41] B Yang R Bao and H Han ldquoRobust hybrid control based onPD and novel CMAC with improved architecture and learningscheme for electric load simulatorrdquo IEEE Transactions onIndustrial Electronics vol 61 no 10 pp 5271ndash5279 2014

[42] S Jagannathan and F L Lewis ldquoMultilayer discrete-timeneural-net controller with guaranteed performancerdquo IEEETransactions on Neural Networks and Learning Systems vol 7no 1 pp 107ndash130 1996

[43] S S Ge and J Wang ldquoRobust adaptive neural control for a classof perturbed strict feedback nonlinear systemsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no6 pp 1409ndash1419 2002

[44] Y H Kim F L Lewis and C T Abdallah ldquoA dynamic recurrentneural-network-based adaptive observer for a class of nonlinearsystemsrdquo Automatica vol 33 no 8 pp 1539ndash1543 1997

[45] J-Q Huang and F L Lewis ldquoNeural-network predictive controlfor nonlinear dynamic systems with time-delayrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 14 no 2pp 377ndash389 2003

[46] S S Ge F Hong and T H Lee ldquoAdaptive neural control ofnonlinear time-delay systems with unknown virtual controlcoefficientsrdquo IEEE Transactions on Systems Man and Cybernet-ics Part B Cybernetics vol 34 no 1 pp 499ndash516 2004

[47] J Na X M Ren and H Huang ldquoTime-delay positive feedbackcontrol for nonlinear time-delay systems with neural networkcompensationrdquo Zidonghua XuebaoActa Automatica Sinica vol34 no 9 pp 1196ndash1202 2008

[48] J Na X Ren C Shang and Y Guo ldquoAdaptive neural networkpredictive control for nonlinear pure feedback systems withinput delayrdquo Journal of Process Control vol 22 no 1 pp 194ndash206 2012

[49] R R Selmic and F L Lewis ldquoNeural-network approximationof piecewise continuous functions application to friction com-pensationrdquo IEEE Transactions on Neural Networks and LearningSystems vol 13 no 3 pp 745ndash751 2002

[50] H Zhang and F L Lewis ldquoAdaptive cooperative trackingcontrol of higher-order nonlinear systems with unknowndynamicsrdquo Automatica vol 48 no 7 pp 1432ndash1439 2012

[51] J Na X Ren and D Zheng ldquoAdaptive control for nonlinearpure-feedback systems with high-order sliding mode observerrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 24 no 3 pp 370ndash382 2013

[52] J Na Q Chen X Ren and Y Guo ldquoAdaptive prescribedperformancemotion control of servomechanisms with frictioncompensationrdquo IEEE Transactions on Industrial Electronics vol61 no 1 pp 486ndash494 2014

[53] J Na X Ren G Herrmann and Z Qiao ldquoAdaptive neuraldynamic surface control for servo systems with unknown dead-zonerdquoControl Engineering Practice vol 19 no 11 pp 1328ndash13432011

[54] G Li J Na D P Stoten and X Ren ldquoAdaptive neural networkfeedforward control for dynamically substructured systemsrdquoIEEE Transactions on Control Systems Technology vol 22 no3 pp 944ndash954 2014

8 Complexity

Positon control Robotic arm Inverse

kinematics

Forward kinematics

Optimal critic learning

EnvironmentImpedance

model

xd

xrqr

x q

f

minus

+

Figure 6 Block of the learning impedence control

Since the control objective is to guarantee the estimationof both robot dynamics and the cost function Υ(119905) theadaptive law is selected as follows

119882119879119886119894 = minus120590119886 (119879119886119894119878119886 + 119896119903Υ) 119878119886 (18)

where 120590119886 and 119896119903 are positive constantsOn the other hand as a fundamental element of the next-

generation robots the human-robot collaboration (HRC) hasbeen widely studied by roboticists and NN is employed inHRCwith its powerful learning ability In [105] theNNswereemployed to estimate the human partnerrsquos motion intentionin human-robot collaboration such that the robot was able toactively follow its human partner To adjust the robotrsquos role tolead or to follow according to the humanrsquos intention gametheory was employed for fundamental analysis of human-robot interaction and an adaptation law was developed in[106] Policy iteration combining with NN was adopted toprovide a rigorous solution to the problem of the systemequilibrium in human-robot interaction [107]

44 NN Based Robot Cognitive Control According to thepredictive processing theory [108] the human brain is alwaysactively anticipating the incoming sensorimotor informationThis process exists because the living beings exhibit latenciesdue to neural processing delays and a limited bandwidthin their sensorimotor processing To compensate for sucha delay in human brain neural feedback signals (includinglateral and top-down connections)modulate the neural activ-ities via inhibitory or excitatory connections by influencingthe neuronal population coding of the bottom-up sensory-driven signals in the perception-action system Similarlyin robotic systems it is claimed that such a delay and alimited bandwidth also can be compensated by the predictivefunctions learnt by recurrent neural models Such a learningprocess can be done via only visual processing [109] or in theloop of perception and action [110]

Based on the hierarchical sensorimotor integration the-ory which advocates that action and perception are inter-twined by sharing the same representational basis [111] therepresentation on different levels of sensory perception doesnot explicitly represent actions instead there is an encodingof the possible future percept which is learnt from priorsensorimotor knowledge

In the Bayesian once this perception and action linkshave been established after learning these perception-action

associations in this architecture allow the following opera-tions

First these associations allow predicting the perceptualoutcome of given actions by means of the forward models(eg Bayesian model) It can be written as

119875 (119864 | 119860 119868) prop 119875 (119860 | 119864) 119875 (119864 | 119868) (19)

where E estimates the upcoming perception evidence givenan executed action A and other prior information you havealready known in 119868 The term 119875(119864 | 119860 119868) suggests that aprelearntmodel representing the possibility of amotor actionA will be executed given that a (possible) resulting sensoryevidence 119864 is perceived (backward computation)

Second these associations allow selecting an appropri-ate movement given an intended perceptual representationFrom the backward computations introduced in the followingequation a predictive sensorimotor integration occurs

119875 (119860 | 119864 119866) prop 119875 (119864 | 119860) 119875 (119860 | 119866) (20)

where A indicates a particular action selected given the(intended) sensory information 119864 and a goal G Here weassume that onersquos action is only determined by the currentsensory input and the goal

In terms of its hierarchical organization it also allows thisoperation with bidirectional information pathways a lowlevel perception representation can be expressed on a higherlevel with a more complex receptive field and vice versa119890low hArr 119890high This can be realized by the bidirectional deeparchitectures such as [112] Conceptually these operationscan be achieved by extracting statistical regularity shown inFigure 7

Since both perception and action processes can be seenas temporal sequences from the mathematical perspectivethe recurrent networks are Turing-Complete and have alearning capacity to learn time sequences with arbitrarylength [113] if properly trained Furthermore such recurrentconnections can be placed in a hierarchical way in whichthe prediction functions on different layers attempt to predictthe nonlinear time-series in different time-scales [114] Fromthis point the recurrent neural network with parametric biasunits (RNNPB) [115] andmultiple time-scale recurrent neuralnetworks (MTRNN) [116] were applied to predict sequencesby understanding them in various temporal levels

The difference of the temporal levels controls the prop-erties of the different levels of the presentation in the deep

Complexity 9

SMA

M1

Brainstem

Spine

Action

Common coding

Common coding

MST

V5MT

V3

V1

Cognitiveprocesses

Visual perception(dorsal)

Figure 7 The process of cognitive control

recurrent network For instance in the MTRNN network[112] the learning of each neuron follows the updating rule ofclassical firing rate models in which the activity of a neuronis determined by the average firing rate of all the connectedneurons Additionally the neuronal activity is also decayingover time following an updating rule of leaky integratormodel Assuming the i-th MTRNN neuron has the numberof N connections the current membrane potential status ofa neuron can be defined by both the previous activation andthe current synaptic inputs

119906119894119905+1 = (1 minus 1120591119894)119906119894119905 + 1120591119894 [sum119908119894119895119909119895119905] (21)

where 119908119894119895 represents the synaptic weight from the j-thneuron to the i-th neuron 119909119895119905 is the activity of j-th neuronat t-th time-step and 120591 is the time-scale parameter whichdetermines the decay rate of this neuron a larger 120591 meanstheir activities change slowly over time compared with thosewith a smaller time-scale parameter 120591

In [117] the concepts of predictive coding were discussedin detail where the learning generation and recognition ofactions can be conducted by means of the principle of pre-diction error minimization By using the predictive codingthe RNNPB and MTRNN are capable for both generatingown actions and recognizing the same actions performedby others Recently the study on neurorobotics experimentshas shown that the dynamic predictive coding scheme canbe used to address fluctuations in temporal patterns whentraining a recurrent neural network (RNN) model [118]This predictive coding scheme enables organisms to predictperceptual outcomes based on current intentions of actionsto the external environment and to forecast perceptualsequences corresponding to given intention states [118]

Based on this architecture two-layer RNN models wereutilized to extract visual information [119] and to under-stand intentions [120] or emotion status [121] in socialrobotics three-layer RNNmodels were used to integrate and

understand multimodal information for a humanoid iCubrobot [112 122] Moreover the predictive coding frameworkhas been extended to variational Bayes predictive codingMTRNN which can arbitrate between deterministic modeland probabilistic model by setting a metaparameter [123]Such extension could provide significant improvement indealing with noisy fluctuated sensory inputs which robots areexpected to experience in more real world setting In [124]a MTRNN was employed to control a humanoid robot andexperimental results have shown that by using only partialtraining data the control model can achieve generalizationby learning in a lower feature perception level

The hierarchical structure of RNN exhibits a greatlearning capacity to store multimodal information whichis beneficial for the robotic systems to understand and topredict in a complex environment As the future models andapplications the state-of-the-art deep learning techniques orthemotor actions of robotic systems can be further integratedinto this predictive architecture

5 Conclusion

In summary great achievements for control design of nonlin-ear system by means of neural networks have been gained inthe last two decades Despite the impossibility in identifyingor listing all the related contributions in this short reviewefforts have been made to summarize the recent progressin the area of NN control and its particular applications inthe robot learning control the robot interaction control andthe robot recognition control In this paper we have shownthat significant progress of NN has been made in control ofthe nonlinear systems in solving the optimization problemin approximating the system dynamics in dealing with theinput nonlinearities in human-robot interaction and in thepattern recognition All these developments accompany notonly the development of techniques in control and advancedmanufactures but also theatrical progress in constructingand developing the neural networks Although huge efforts

10 Complexity

have been made to embed the NN in practical control sys-tems there is still a large gap between the theory and practiceTo improve the feasibility and usability the evolutionarycomputing theory has been proposed to train the NNs It canautomatically find a near-optimal NN architecture and allowa NN to adapt its learning rule to its environment Howeverthe complex and long training process of the evolutionaryalgorithms deters their practical applications More effortsneed to be made to evolve the NN architecture and NNlearning technique in the control design On the otherhand in human brain the neural activities are modulatedvia inhibitory or excitatory connections by influencing theneuronal population coding of the bottom-up sensory-drivensignals in the perception-action system In this sense howto integrate the sensor-motor information into the networkto make NNs more feasible to adapt to the environment andto resemble the capacity of the human brain deserves furtherinvestigations

In conclusion a brief review on neural networks for thecomplex nonlinear systems is provided with adaptive neuralcontrol NN based dynamic programming evolution com-puting and their practical applications in the robotic fieldsWe believe this area may promote increasing investigationsin both theories and applications And emerging topics likedeep learning [125ndash128] big data [129ndash131] and cloud com-puting may be incorporated into the neural network controlfor complex systems for example deep neural networkscould be used to process massive amounts of unsuperviseddata in complex scenarios neural networks can be helpfulin reducing the data dimensionality and the optimization ofNN training may be employed to enhance the learning andadaptation performance of robots

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

Thisworkwas partially supported by theNational Nature Sci-ence Foundation (NSFC) under Grant 61473120 GuangdongProvincial Natural Science Foundation 2014A030313266International Science and Technology Collaboration Grant2015A050502017 Science andTechnology Planning Project ofGuangzhou 201607010006 State Key Laboratory of Roboticsand System (HIT) Grant SKLRS-2017-KF-13 and the Funda-mental Research Funds for the Central Universities

References

[1] F Gruau D Whitley and L Pyeatt ldquoA comparison betweencellular encoding and direct encoding for genetic neural net-worksrdquo inProceedings of the 1st annual conference on genetic pro-gramming pp 81ndash89 1996

[2] H de Garis ldquoAn artificial brain ATRrsquos CAM-Brain Project aimsto buildevolve an artificial brain with a million neural netmodules inside a trillion cell Cellular Automata Machinerdquo NewGeneration Computing vol 12 no 2 pp 215ndash221 1994

[3] W S McCulloch and W Pitts ldquoA logical calculus of the ideasimmanent in nervous activityrdquoBulletin ofMathematical Biologyvol 5 no 4 pp 115ndash133 1943

[4] C Yang S S Ge C Xiang T Chai and T H Lee ldquoOutput feed-back NN control for two classes of discrete-time systems withunknown control directions in a unified approachrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 19 no 11pp 1873ndash1886 2008

[5] R Kumar R K Aggarwal and J D Sharma ldquoEnergy analysis ofa building using artificial neural network A reviewrdquo Energy andBuildings vol 65 pp 352ndash358 2013

[6] C Alippi C De Russis and V Piuri ldquoA neural-network basedcontrol solution to air-fuel ratio control for automotive fuel-injection systemsrdquo IEEE Transactions on Systems Man andCybernetics Part C Applications and Reviews vol 33 no 2 pp259ndash268 2003

[7] E M Azoff Neural network time series forecasting of financialmarkets John Wiley amp Sons Inc 1994

[8] I Fister P N Suganthan J Fister et al ldquoArtificial neural net-work regression as a local search heuristic for ensemble strate-gies in differential evolutionrdquo Nonlinear Dynamics vol 84 no2 pp 895ndash914 2016

[9] Y Zhang S Li and H Guo ldquoA type of biased consensus-based distributed neural network for path planningrdquoNonlinearDynamics vol 89 no 3 pp 1803ndash1815 2017

[10] X Shi Z Wang and L Han ldquoFinite-time stochastic synchro-nization of time-delay neural networks with noise disturbancerdquoNonlinear Dynamics vol 88 no 4 pp 2747ndash2755 2017

[11] P He and Y Li ldquoHinfin synchronization of coupled reaction-diffusion neural networks with mixed delaysrdquo Complexity vol21 no S2 pp 42ndash53 2016

[12] Z Tu J Cao A Alsaedi F E Alsaadi and T Hayat ldquoGlobalLagrange stability of complex-valued neural networks of neutraltype with time-varying delaysrdquo Complexity vol 21 no S2 pp438ndash450 2016

[13] C Wang S Guo and Y Xu ldquoFormation of autapse connectedto neuron and its biological functionrdquo Complexity vol 2017Article ID 5436737 9 pages 2017

[14] J D J Rubio I Elias D R Cruz and J Pacheco ldquoUniform stableradial basis function neural network for the prediction in twomechatronic processesrdquo Neurocomputing vol 227 pp 122ndash1302017

[15] J d Rubio ldquoUSNFIS Uniform stable neuro fuzzy inferencesystemrdquo Neurocomputing vol 262 pp 57ndash66 2017

[16] Q Liu J Yin V C M Leung J-H Zhai Z Cai and J LinldquoApplying a new localized generalization error model to designneural networks trained with extreme learning machinerdquo Neu-ral Computing and Applications pp 1ndash8 2014

[17] R J de Jesus ldquoInterpolation neural network model of a manu-factured wind turbinerdquoNeural Computing and Applications pp1ndash12 2016

[18] C Mu and D Wang ldquoNeural-network-based adaptive guaran-teed cost control of nonlinear dynamical systems with matcheduncertaintiesrdquo Neurocomputing vol 245 pp 46ndash54 2017

[19] Z Lin DMa J Meng and L Chen ldquoRelative ordering learningin spiking neural network for pattern recognitionrdquo Neuro-computing 2017

[20] J Yu J Sang and X Gao ldquoMachine learning and signal pro-cessing for big multimedia analysisrdquo Neurocomputing vol 257pp 1ndash4 2017

Complexity 11

[21] T Zhang S S Ge and C C Hang ldquoAdaptive neural networkcontrol for strict-feedback nonlinear systems using backstep-ping designrdquo Automatica vol 36 no 12 pp 1835ndash1846 2000

[22] S S Ge and T Zhang ldquoNeural-network control of nonaffinenonlinear system with zero dynamics by state and outputfeedbackrdquo IEEE Transactions on Neural Networks and LearningSystems vol 14 no 4 pp 900ndash918 2003

[23] S S Ge C Yang and T H Lee ldquoAdaptive predictive controlusing neural network for a class of pure-feedback systems in dis-crete timerdquo IEEE Transactions on Neural Networks and LearningSystems vol 19 no 9 pp 1599ndash1614 2008

[24] F W Lewis S Jagannathan and A Yesildirak Neural networkcontrol of robotmanipulators and non-linear systems CRCPress1998

[25] S Jagannathan and F L Lewis ldquoIdentification of nonlineardynamical systems using multilayered neural networksrdquo Auto-matica vol 32 no 12 pp 1707ndash1712 1996

[26] D Vrabie and F Lewis ldquoNeural network approach tocontinuous-time direct adaptive optimal control for partiallyunknown nonlinear systemsrdquo Neural Networks vol 22 no 3pp 237ndash246 2009

[27] C Yang S S Ge and T H Lee ldquoOutput feedback adaptive con-trol of a class of nonlinear discrete-time systems with unknowncontrol directionsrdquo Automatica vol 45 no 1 pp 270ndash2762009

[28] C Yang Z Li and J Li ldquoTrajectory planning and optimizedadaptive control for a class of wheeled inverted pendulumvehicle modelsrdquo IEEE Transactions on Cybernetics vol 43 no 1pp 24ndash36 2013

[29] Y Jiang C Yang and H Ma ldquoA review of fuzzy logic andneural network based intelligent control design for discrete-time systemsrdquo Discrete Dynamics in Nature and Society ArticleID 7217364 Art ID 7217364 11 pages 2016

[30] Y Jiang C Yang S-L Dai and B Ren ldquoDeterministic learningenhanced neutral network control of unmanned helicopterrdquoInternational Journal of Advanced Robotic Systems vol 13 no6 pp 1ndash12 2016

[31] Y Jiang Z Liu C Chen and Y Zhang ldquoAdaptive robust fuzzycontrol for dual arm robot with unknown input deadzonenonlinearityrdquo Nonlinear Dynamics vol 81 no 3 pp 1301ndash13142015

[32] httpsstemcellthailandorgneurons-definition-function-neurotransmitters

[33] MDefoort T Floquet A Kokosy andW Perruquetti ldquoSliding-mode formation control for cooperative autonomous mobilerobotsrdquo IEEE Transactions on Industrial Electronics vol 55 no11 pp 3944ndash3953 2008

[34] X Liu C Yang Z Chen MWang and C Su ldquoNeuro-adaptiveobserver based control of flexible joint robotrdquoNeurocomputing2017

[35] F Hamerlain T Floquet and W Perruquetti ldquoExperimentaltests of a slidingmode controller for trajectory tracking of a car-like mobile robotrdquo Robotica vol 32 no 1 pp 63ndash76 2014

[36] R J de Jesus ldquoDiscrete time control based in neural networksfor pendulumsrdquo Applied Soft Computing 2017

[37] Y Pan M J Er T Sun B Xu and H Yu ldquoAdaptive fuzzy PDcontrol with stable Hinfin tracking guaranteerdquo Neurocomputingvol 237 pp 71ndash78 2017

[38] R J de Jesus ldquoAdaptive least square control in discrete time ofrobotic armsrdquo Soft Computing vol 19 no 12 pp 3665ndash36762015

[39] S Commuri S Jagannathan and F L Lewis ldquoCMAC neuralnetwork control of robot manipulatorsrdquo Journal of RoboticSystems vol 14 no 6 pp 465ndash482 1997

[40] J S Albus ldquoTheoretical and experimental aspects of a cerebellarmodelrdquoDevelopmental Disabilities Research Reviews vol 17 pp93ndash101 1972

[41] B Yang R Bao and H Han ldquoRobust hybrid control based onPD and novel CMAC with improved architecture and learningscheme for electric load simulatorrdquo IEEE Transactions onIndustrial Electronics vol 61 no 10 pp 5271ndash5279 2014

[42] S Jagannathan and F L Lewis ldquoMultilayer discrete-timeneural-net controller with guaranteed performancerdquo IEEETransactions on Neural Networks and Learning Systems vol 7no 1 pp 107ndash130 1996

[43] S S Ge and J Wang ldquoRobust adaptive neural control for a classof perturbed strict feedback nonlinear systemsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no6 pp 1409ndash1419 2002

[44] Y H Kim F L Lewis and C T Abdallah ldquoA dynamic recurrentneural-network-based adaptive observer for a class of nonlinearsystemsrdquo Automatica vol 33 no 8 pp 1539ndash1543 1997

[45] J-Q Huang and F L Lewis ldquoNeural-network predictive controlfor nonlinear dynamic systems with time-delayrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 14 no 2pp 377ndash389 2003

[46] S S Ge F Hong and T H Lee ldquoAdaptive neural control ofnonlinear time-delay systems with unknown virtual controlcoefficientsrdquo IEEE Transactions on Systems Man and Cybernet-ics Part B Cybernetics vol 34 no 1 pp 499ndash516 2004

[47] J Na X M Ren and H Huang ldquoTime-delay positive feedbackcontrol for nonlinear time-delay systems with neural networkcompensationrdquo Zidonghua XuebaoActa Automatica Sinica vol34 no 9 pp 1196ndash1202 2008

[48] J Na X Ren C Shang and Y Guo ldquoAdaptive neural networkpredictive control for nonlinear pure feedback systems withinput delayrdquo Journal of Process Control vol 22 no 1 pp 194ndash206 2012

[49] R R Selmic and F L Lewis ldquoNeural-network approximationof piecewise continuous functions application to friction com-pensationrdquo IEEE Transactions on Neural Networks and LearningSystems vol 13 no 3 pp 745ndash751 2002

[50] H Zhang and F L Lewis ldquoAdaptive cooperative trackingcontrol of higher-order nonlinear systems with unknowndynamicsrdquo Automatica vol 48 no 7 pp 1432ndash1439 2012

[51] J Na X Ren and D Zheng ldquoAdaptive control for nonlinearpure-feedback systems with high-order sliding mode observerrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 24 no 3 pp 370ndash382 2013

[52] J Na Q Chen X Ren and Y Guo ldquoAdaptive prescribedperformancemotion control of servomechanisms with frictioncompensationrdquo IEEE Transactions on Industrial Electronics vol61 no 1 pp 486ndash494 2014

[53] J Na X Ren G Herrmann and Z Qiao ldquoAdaptive neuraldynamic surface control for servo systems with unknown dead-zonerdquoControl Engineering Practice vol 19 no 11 pp 1328ndash13432011

[54] G Li J Na D P Stoten and X Ren ldquoAdaptive neural networkfeedforward control for dynamically substructured systemsrdquoIEEE Transactions on Control Systems Technology vol 22 no3 pp 944ndash954 2014

Complexity 9

SMA

M1

Brainstem

Spine

Action

Common coding

Common coding

MST

V5MT

V3

V1

Cognitiveprocesses

Visual perception(dorsal)

Figure 7 The process of cognitive control

recurrent network For instance in the MTRNN network[112] the learning of each neuron follows the updating rule ofclassical firing rate models in which the activity of a neuronis determined by the average firing rate of all the connectedneurons Additionally the neuronal activity is also decayingover time following an updating rule of leaky integratormodel Assuming the i-th MTRNN neuron has the numberof N connections the current membrane potential status ofa neuron can be defined by both the previous activation andthe current synaptic inputs

119906119894119905+1 = (1 minus 1120591119894)119906119894119905 + 1120591119894 [sum119908119894119895119909119895119905] (21)

where 119908119894119895 represents the synaptic weight from the j-thneuron to the i-th neuron 119909119895119905 is the activity of j-th neuronat t-th time-step and 120591 is the time-scale parameter whichdetermines the decay rate of this neuron a larger 120591 meanstheir activities change slowly over time compared with thosewith a smaller time-scale parameter 120591

In [117] the concepts of predictive coding were discussedin detail where the learning generation and recognition ofactions can be conducted by means of the principle of pre-diction error minimization By using the predictive codingthe RNNPB and MTRNN are capable for both generatingown actions and recognizing the same actions performedby others Recently the study on neurorobotics experimentshas shown that the dynamic predictive coding scheme canbe used to address fluctuations in temporal patterns whentraining a recurrent neural network (RNN) model [118]This predictive coding scheme enables organisms to predictperceptual outcomes based on current intentions of actionsto the external environment and to forecast perceptualsequences corresponding to given intention states [118]

Based on this architecture two-layer RNN models wereutilized to extract visual information [119] and to under-stand intentions [120] or emotion status [121] in socialrobotics three-layer RNNmodels were used to integrate and

understand multimodal information for a humanoid iCubrobot [112 122] Moreover the predictive coding frameworkhas been extended to variational Bayes predictive codingMTRNN which can arbitrate between deterministic modeland probabilistic model by setting a metaparameter [123]Such extension could provide significant improvement indealing with noisy fluctuated sensory inputs which robots areexpected to experience in more real world setting In [124]a MTRNN was employed to control a humanoid robot andexperimental results have shown that by using only partialtraining data the control model can achieve generalizationby learning in a lower feature perception level

The hierarchical structure of RNN exhibits a greatlearning capacity to store multimodal information whichis beneficial for the robotic systems to understand and topredict in a complex environment As the future models andapplications the state-of-the-art deep learning techniques orthemotor actions of robotic systems can be further integratedinto this predictive architecture

5 Conclusion

In summary great achievements for control design of nonlin-ear system by means of neural networks have been gained inthe last two decades Despite the impossibility in identifyingor listing all the related contributions in this short reviewefforts have been made to summarize the recent progressin the area of NN control and its particular applications inthe robot learning control the robot interaction control andthe robot recognition control In this paper we have shownthat significant progress of NN has been made in control ofthe nonlinear systems in solving the optimization problemin approximating the system dynamics in dealing with theinput nonlinearities in human-robot interaction and in thepattern recognition All these developments accompany notonly the development of techniques in control and advancedmanufactures but also theatrical progress in constructingand developing the neural networks Although huge efforts

10 Complexity

have been made to embed the NN in practical control sys-tems there is still a large gap between the theory and practiceTo improve the feasibility and usability the evolutionarycomputing theory has been proposed to train the NNs It canautomatically find a near-optimal NN architecture and allowa NN to adapt its learning rule to its environment Howeverthe complex and long training process of the evolutionaryalgorithms deters their practical applications More effortsneed to be made to evolve the NN architecture and NNlearning technique in the control design On the otherhand in human brain the neural activities are modulatedvia inhibitory or excitatory connections by influencing theneuronal population coding of the bottom-up sensory-drivensignals in the perception-action system In this sense howto integrate the sensor-motor information into the networkto make NNs more feasible to adapt to the environment andto resemble the capacity of the human brain deserves furtherinvestigations

In conclusion a brief review on neural networks for thecomplex nonlinear systems is provided with adaptive neuralcontrol NN based dynamic programming evolution com-puting and their practical applications in the robotic fieldsWe believe this area may promote increasing investigationsin both theories and applications And emerging topics likedeep learning [125ndash128] big data [129ndash131] and cloud com-puting may be incorporated into the neural network controlfor complex systems for example deep neural networkscould be used to process massive amounts of unsuperviseddata in complex scenarios neural networks can be helpfulin reducing the data dimensionality and the optimization ofNN training may be employed to enhance the learning andadaptation performance of robots

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

Thisworkwas partially supported by theNational Nature Sci-ence Foundation (NSFC) under Grant 61473120 GuangdongProvincial Natural Science Foundation 2014A030313266International Science and Technology Collaboration Grant2015A050502017 Science andTechnology Planning Project ofGuangzhou 201607010006 State Key Laboratory of Roboticsand System (HIT) Grant SKLRS-2017-KF-13 and the Funda-mental Research Funds for the Central Universities

References

[1] F Gruau D Whitley and L Pyeatt ldquoA comparison betweencellular encoding and direct encoding for genetic neural net-worksrdquo inProceedings of the 1st annual conference on genetic pro-gramming pp 81ndash89 1996

[2] H de Garis ldquoAn artificial brain ATRrsquos CAM-Brain Project aimsto buildevolve an artificial brain with a million neural netmodules inside a trillion cell Cellular Automata Machinerdquo NewGeneration Computing vol 12 no 2 pp 215ndash221 1994

[3] W S McCulloch and W Pitts ldquoA logical calculus of the ideasimmanent in nervous activityrdquoBulletin ofMathematical Biologyvol 5 no 4 pp 115ndash133 1943

[4] C Yang S S Ge C Xiang T Chai and T H Lee ldquoOutput feed-back NN control for two classes of discrete-time systems withunknown control directions in a unified approachrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 19 no 11pp 1873ndash1886 2008

[5] R Kumar R K Aggarwal and J D Sharma ldquoEnergy analysis ofa building using artificial neural network A reviewrdquo Energy andBuildings vol 65 pp 352ndash358 2013

[6] C Alippi C De Russis and V Piuri ldquoA neural-network basedcontrol solution to air-fuel ratio control for automotive fuel-injection systemsrdquo IEEE Transactions on Systems Man andCybernetics Part C Applications and Reviews vol 33 no 2 pp259ndash268 2003

[7] E M Azoff Neural network time series forecasting of financialmarkets John Wiley amp Sons Inc 1994

[8] I Fister P N Suganthan J Fister et al ldquoArtificial neural net-work regression as a local search heuristic for ensemble strate-gies in differential evolutionrdquo Nonlinear Dynamics vol 84 no2 pp 895ndash914 2016

[9] Y Zhang S Li and H Guo ldquoA type of biased consensus-based distributed neural network for path planningrdquoNonlinearDynamics vol 89 no 3 pp 1803ndash1815 2017

[10] X Shi Z Wang and L Han ldquoFinite-time stochastic synchro-nization of time-delay neural networks with noise disturbancerdquoNonlinear Dynamics vol 88 no 4 pp 2747ndash2755 2017

[11] P He and Y Li ldquoHinfin synchronization of coupled reaction-diffusion neural networks with mixed delaysrdquo Complexity vol21 no S2 pp 42ndash53 2016

[12] Z Tu J Cao A Alsaedi F E Alsaadi and T Hayat ldquoGlobalLagrange stability of complex-valued neural networks of neutraltype with time-varying delaysrdquo Complexity vol 21 no S2 pp438ndash450 2016

[13] C Wang S Guo and Y Xu ldquoFormation of autapse connectedto neuron and its biological functionrdquo Complexity vol 2017Article ID 5436737 9 pages 2017

[14] J D J Rubio I Elias D R Cruz and J Pacheco ldquoUniform stableradial basis function neural network for the prediction in twomechatronic processesrdquo Neurocomputing vol 227 pp 122ndash1302017

[15] J d Rubio ldquoUSNFIS Uniform stable neuro fuzzy inferencesystemrdquo Neurocomputing vol 262 pp 57ndash66 2017

[16] Q Liu J Yin V C M Leung J-H Zhai Z Cai and J LinldquoApplying a new localized generalization error model to designneural networks trained with extreme learning machinerdquo Neu-ral Computing and Applications pp 1ndash8 2014

[17] R J de Jesus ldquoInterpolation neural network model of a manu-factured wind turbinerdquoNeural Computing and Applications pp1ndash12 2016

[18] C Mu and D Wang ldquoNeural-network-based adaptive guaran-teed cost control of nonlinear dynamical systems with matcheduncertaintiesrdquo Neurocomputing vol 245 pp 46ndash54 2017

[19] Z Lin DMa J Meng and L Chen ldquoRelative ordering learningin spiking neural network for pattern recognitionrdquo Neuro-computing 2017

[20] J Yu J Sang and X Gao ldquoMachine learning and signal pro-cessing for big multimedia analysisrdquo Neurocomputing vol 257pp 1ndash4 2017

Complexity 11

[21] T Zhang S S Ge and C C Hang ldquoAdaptive neural networkcontrol for strict-feedback nonlinear systems using backstep-ping designrdquo Automatica vol 36 no 12 pp 1835ndash1846 2000

[22] S S Ge and T Zhang ldquoNeural-network control of nonaffinenonlinear system with zero dynamics by state and outputfeedbackrdquo IEEE Transactions on Neural Networks and LearningSystems vol 14 no 4 pp 900ndash918 2003

[23] S S Ge C Yang and T H Lee ldquoAdaptive predictive controlusing neural network for a class of pure-feedback systems in dis-crete timerdquo IEEE Transactions on Neural Networks and LearningSystems vol 19 no 9 pp 1599ndash1614 2008

[24] F W Lewis S Jagannathan and A Yesildirak Neural networkcontrol of robotmanipulators and non-linear systems CRCPress1998

[25] S Jagannathan and F L Lewis ldquoIdentification of nonlineardynamical systems using multilayered neural networksrdquo Auto-matica vol 32 no 12 pp 1707ndash1712 1996

[26] D Vrabie and F Lewis ldquoNeural network approach tocontinuous-time direct adaptive optimal control for partiallyunknown nonlinear systemsrdquo Neural Networks vol 22 no 3pp 237ndash246 2009

[27] C Yang S S Ge and T H Lee ldquoOutput feedback adaptive con-trol of a class of nonlinear discrete-time systems with unknowncontrol directionsrdquo Automatica vol 45 no 1 pp 270ndash2762009

[28] C Yang Z Li and J Li ldquoTrajectory planning and optimizedadaptive control for a class of wheeled inverted pendulumvehicle modelsrdquo IEEE Transactions on Cybernetics vol 43 no 1pp 24ndash36 2013

[29] Y Jiang C Yang and H Ma ldquoA review of fuzzy logic andneural network based intelligent control design for discrete-time systemsrdquo Discrete Dynamics in Nature and Society ArticleID 7217364 Art ID 7217364 11 pages 2016

[30] Y Jiang C Yang S-L Dai and B Ren ldquoDeterministic learningenhanced neutral network control of unmanned helicopterrdquoInternational Journal of Advanced Robotic Systems vol 13 no6 pp 1ndash12 2016

[31] Y Jiang Z Liu C Chen and Y Zhang ldquoAdaptive robust fuzzycontrol for dual arm robot with unknown input deadzonenonlinearityrdquo Nonlinear Dynamics vol 81 no 3 pp 1301ndash13142015

[32] httpsstemcellthailandorgneurons-definition-function-neurotransmitters

[33] MDefoort T Floquet A Kokosy andW Perruquetti ldquoSliding-mode formation control for cooperative autonomous mobilerobotsrdquo IEEE Transactions on Industrial Electronics vol 55 no11 pp 3944ndash3953 2008

[34] X Liu C Yang Z Chen MWang and C Su ldquoNeuro-adaptiveobserver based control of flexible joint robotrdquoNeurocomputing2017

[35] F Hamerlain T Floquet and W Perruquetti ldquoExperimentaltests of a slidingmode controller for trajectory tracking of a car-like mobile robotrdquo Robotica vol 32 no 1 pp 63ndash76 2014

[36] R J de Jesus ldquoDiscrete time control based in neural networksfor pendulumsrdquo Applied Soft Computing 2017

[37] Y Pan M J Er T Sun B Xu and H Yu ldquoAdaptive fuzzy PDcontrol with stable Hinfin tracking guaranteerdquo Neurocomputingvol 237 pp 71ndash78 2017

[38] R J de Jesus ldquoAdaptive least square control in discrete time ofrobotic armsrdquo Soft Computing vol 19 no 12 pp 3665ndash36762015

[39] S Commuri S Jagannathan and F L Lewis ldquoCMAC neuralnetwork control of robot manipulatorsrdquo Journal of RoboticSystems vol 14 no 6 pp 465ndash482 1997

[40] J S Albus ldquoTheoretical and experimental aspects of a cerebellarmodelrdquoDevelopmental Disabilities Research Reviews vol 17 pp93ndash101 1972

[41] B Yang R Bao and H Han ldquoRobust hybrid control based onPD and novel CMAC with improved architecture and learningscheme for electric load simulatorrdquo IEEE Transactions onIndustrial Electronics vol 61 no 10 pp 5271ndash5279 2014

[42] S Jagannathan and F L Lewis ldquoMultilayer discrete-timeneural-net controller with guaranteed performancerdquo IEEETransactions on Neural Networks and Learning Systems vol 7no 1 pp 107ndash130 1996

[43] S S Ge and J Wang ldquoRobust adaptive neural control for a classof perturbed strict feedback nonlinear systemsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no6 pp 1409ndash1419 2002

[44] Y H Kim F L Lewis and C T Abdallah ldquoA dynamic recurrentneural-network-based adaptive observer for a class of nonlinearsystemsrdquo Automatica vol 33 no 8 pp 1539ndash1543 1997

[45] J-Q Huang and F L Lewis ldquoNeural-network predictive controlfor nonlinear dynamic systems with time-delayrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 14 no 2pp 377ndash389 2003

[46] S S Ge F Hong and T H Lee ldquoAdaptive neural control ofnonlinear time-delay systems with unknown virtual controlcoefficientsrdquo IEEE Transactions on Systems Man and Cybernet-ics Part B Cybernetics vol 34 no 1 pp 499ndash516 2004

[47] J Na X M Ren and H Huang ldquoTime-delay positive feedbackcontrol for nonlinear time-delay systems with neural networkcompensationrdquo Zidonghua XuebaoActa Automatica Sinica vol34 no 9 pp 1196ndash1202 2008

[48] J Na X Ren C Shang and Y Guo ldquoAdaptive neural networkpredictive control for nonlinear pure feedback systems withinput delayrdquo Journal of Process Control vol 22 no 1 pp 194ndash206 2012

[49] R R Selmic and F L Lewis ldquoNeural-network approximationof piecewise continuous functions application to friction com-pensationrdquo IEEE Transactions on Neural Networks and LearningSystems vol 13 no 3 pp 745ndash751 2002

[50] H Zhang and F L Lewis ldquoAdaptive cooperative trackingcontrol of higher-order nonlinear systems with unknowndynamicsrdquo Automatica vol 48 no 7 pp 1432ndash1439 2012

[51] J Na X Ren and D Zheng ldquoAdaptive control for nonlinearpure-feedback systems with high-order sliding mode observerrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 24 no 3 pp 370ndash382 2013

[52] J Na Q Chen X Ren and Y Guo ldquoAdaptive prescribedperformancemotion control of servomechanisms with frictioncompensationrdquo IEEE Transactions on Industrial Electronics vol61 no 1 pp 486ndash494 2014

[53] J Na X Ren G Herrmann and Z Qiao ldquoAdaptive neuraldynamic surface control for servo systems with unknown dead-zonerdquoControl Engineering Practice vol 19 no 11 pp 1328ndash13432011

[54] G Li J Na D P Stoten and X Ren ldquoAdaptive neural networkfeedforward control for dynamically substructured systemsrdquoIEEE Transactions on Control Systems Technology vol 22 no3 pp 944ndash954 2014

10 Complexity

have been made to embed the NN in practical control sys-tems there is still a large gap between the theory and practiceTo improve the feasibility and usability the evolutionarycomputing theory has been proposed to train the NNs It canautomatically find a near-optimal NN architecture and allowa NN to adapt its learning rule to its environment Howeverthe complex and long training process of the evolutionaryalgorithms deters their practical applications More effortsneed to be made to evolve the NN architecture and NNlearning technique in the control design On the otherhand in human brain the neural activities are modulatedvia inhibitory or excitatory connections by influencing theneuronal population coding of the bottom-up sensory-drivensignals in the perception-action system In this sense howto integrate the sensor-motor information into the networkto make NNs more feasible to adapt to the environment andto resemble the capacity of the human brain deserves furtherinvestigations

In conclusion a brief review on neural networks for thecomplex nonlinear systems is provided with adaptive neuralcontrol NN based dynamic programming evolution com-puting and their practical applications in the robotic fieldsWe believe this area may promote increasing investigationsin both theories and applications And emerging topics likedeep learning [125ndash128] big data [129ndash131] and cloud com-puting may be incorporated into the neural network controlfor complex systems for example deep neural networkscould be used to process massive amounts of unsuperviseddata in complex scenarios neural networks can be helpfulin reducing the data dimensionality and the optimization ofNN training may be employed to enhance the learning andadaptation performance of robots

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

Thisworkwas partially supported by theNational Nature Sci-ence Foundation (NSFC) under Grant 61473120 GuangdongProvincial Natural Science Foundation 2014A030313266International Science and Technology Collaboration Grant2015A050502017 Science andTechnology Planning Project ofGuangzhou 201607010006 State Key Laboratory of Roboticsand System (HIT) Grant SKLRS-2017-KF-13 and the Funda-mental Research Funds for the Central Universities

References

[1] F Gruau D Whitley and L Pyeatt ldquoA comparison betweencellular encoding and direct encoding for genetic neural net-worksrdquo inProceedings of the 1st annual conference on genetic pro-gramming pp 81ndash89 1996

[2] H de Garis ldquoAn artificial brain ATRrsquos CAM-Brain Project aimsto buildevolve an artificial brain with a million neural netmodules inside a trillion cell Cellular Automata Machinerdquo NewGeneration Computing vol 12 no 2 pp 215ndash221 1994

[3] W S McCulloch and W Pitts ldquoA logical calculus of the ideasimmanent in nervous activityrdquoBulletin ofMathematical Biologyvol 5 no 4 pp 115ndash133 1943

[4] C Yang S S Ge C Xiang T Chai and T H Lee ldquoOutput feed-back NN control for two classes of discrete-time systems withunknown control directions in a unified approachrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 19 no 11pp 1873ndash1886 2008

[5] R Kumar R K Aggarwal and J D Sharma ldquoEnergy analysis ofa building using artificial neural network A reviewrdquo Energy andBuildings vol 65 pp 352ndash358 2013

[6] C Alippi C De Russis and V Piuri ldquoA neural-network basedcontrol solution to air-fuel ratio control for automotive fuel-injection systemsrdquo IEEE Transactions on Systems Man andCybernetics Part C Applications and Reviews vol 33 no 2 pp259ndash268 2003

[7] E M Azoff Neural network time series forecasting of financialmarkets John Wiley amp Sons Inc 1994

[8] I Fister P N Suganthan J Fister et al ldquoArtificial neural net-work regression as a local search heuristic for ensemble strate-gies in differential evolutionrdquo Nonlinear Dynamics vol 84 no2 pp 895ndash914 2016

[9] Y Zhang S Li and H Guo ldquoA type of biased consensus-based distributed neural network for path planningrdquoNonlinearDynamics vol 89 no 3 pp 1803ndash1815 2017

[10] X Shi Z Wang and L Han ldquoFinite-time stochastic synchro-nization of time-delay neural networks with noise disturbancerdquoNonlinear Dynamics vol 88 no 4 pp 2747ndash2755 2017

[11] P He and Y Li ldquoHinfin synchronization of coupled reaction-diffusion neural networks with mixed delaysrdquo Complexity vol21 no S2 pp 42ndash53 2016

[12] Z Tu J Cao A Alsaedi F E Alsaadi and T Hayat ldquoGlobalLagrange stability of complex-valued neural networks of neutraltype with time-varying delaysrdquo Complexity vol 21 no S2 pp438ndash450 2016

[13] C Wang S Guo and Y Xu ldquoFormation of autapse connectedto neuron and its biological functionrdquo Complexity vol 2017Article ID 5436737 9 pages 2017

[14] J D J Rubio I Elias D R Cruz and J Pacheco ldquoUniform stableradial basis function neural network for the prediction in twomechatronic processesrdquo Neurocomputing vol 227 pp 122ndash1302017

[15] J d Rubio ldquoUSNFIS Uniform stable neuro fuzzy inferencesystemrdquo Neurocomputing vol 262 pp 57ndash66 2017

[16] Q Liu J Yin V C M Leung J-H Zhai Z Cai and J LinldquoApplying a new localized generalization error model to designneural networks trained with extreme learning machinerdquo Neu-ral Computing and Applications pp 1ndash8 2014

[17] R J de Jesus ldquoInterpolation neural network model of a manu-factured wind turbinerdquoNeural Computing and Applications pp1ndash12 2016

[18] C Mu and D Wang ldquoNeural-network-based adaptive guaran-teed cost control of nonlinear dynamical systems with matcheduncertaintiesrdquo Neurocomputing vol 245 pp 46ndash54 2017

[19] Z Lin DMa J Meng and L Chen ldquoRelative ordering learningin spiking neural network for pattern recognitionrdquo Neuro-computing 2017

[20] J Yu J Sang and X Gao ldquoMachine learning and signal pro-cessing for big multimedia analysisrdquo Neurocomputing vol 257pp 1ndash4 2017

Complexity 11

[21] T Zhang S S Ge and C C Hang ldquoAdaptive neural networkcontrol for strict-feedback nonlinear systems using backstep-ping designrdquo Automatica vol 36 no 12 pp 1835ndash1846 2000

[22] S S Ge and T Zhang ldquoNeural-network control of nonaffinenonlinear system with zero dynamics by state and outputfeedbackrdquo IEEE Transactions on Neural Networks and LearningSystems vol 14 no 4 pp 900ndash918 2003

[23] S S Ge C Yang and T H Lee ldquoAdaptive predictive controlusing neural network for a class of pure-feedback systems in dis-crete timerdquo IEEE Transactions on Neural Networks and LearningSystems vol 19 no 9 pp 1599ndash1614 2008

[24] F W Lewis S Jagannathan and A Yesildirak Neural networkcontrol of robotmanipulators and non-linear systems CRCPress1998

[25] S Jagannathan and F L Lewis ldquoIdentification of nonlineardynamical systems using multilayered neural networksrdquo Auto-matica vol 32 no 12 pp 1707ndash1712 1996

[26] D Vrabie and F Lewis ldquoNeural network approach tocontinuous-time direct adaptive optimal control for partiallyunknown nonlinear systemsrdquo Neural Networks vol 22 no 3pp 237ndash246 2009

[27] C Yang S S Ge and T H Lee ldquoOutput feedback adaptive con-trol of a class of nonlinear discrete-time systems with unknowncontrol directionsrdquo Automatica vol 45 no 1 pp 270ndash2762009

[28] C Yang Z Li and J Li ldquoTrajectory planning and optimizedadaptive control for a class of wheeled inverted pendulumvehicle modelsrdquo IEEE Transactions on Cybernetics vol 43 no 1pp 24ndash36 2013

[29] Y Jiang C Yang and H Ma ldquoA review of fuzzy logic andneural network based intelligent control design for discrete-time systemsrdquo Discrete Dynamics in Nature and Society ArticleID 7217364 Art ID 7217364 11 pages 2016

[30] Y Jiang C Yang S-L Dai and B Ren ldquoDeterministic learningenhanced neutral network control of unmanned helicopterrdquoInternational Journal of Advanced Robotic Systems vol 13 no6 pp 1ndash12 2016

[31] Y Jiang Z Liu C Chen and Y Zhang ldquoAdaptive robust fuzzycontrol for dual arm robot with unknown input deadzonenonlinearityrdquo Nonlinear Dynamics vol 81 no 3 pp 1301ndash13142015

[32] httpsstemcellthailandorgneurons-definition-function-neurotransmitters

[33] MDefoort T Floquet A Kokosy andW Perruquetti ldquoSliding-mode formation control for cooperative autonomous mobilerobotsrdquo IEEE Transactions on Industrial Electronics vol 55 no11 pp 3944ndash3953 2008

[34] X Liu C Yang Z Chen MWang and C Su ldquoNeuro-adaptiveobserver based control of flexible joint robotrdquoNeurocomputing2017

[35] F Hamerlain T Floquet and W Perruquetti ldquoExperimentaltests of a slidingmode controller for trajectory tracking of a car-like mobile robotrdquo Robotica vol 32 no 1 pp 63ndash76 2014

[36] R J de Jesus ldquoDiscrete time control based in neural networksfor pendulumsrdquo Applied Soft Computing 2017

[37] Y Pan M J Er T Sun B Xu and H Yu ldquoAdaptive fuzzy PDcontrol with stable Hinfin tracking guaranteerdquo Neurocomputingvol 237 pp 71ndash78 2017

[38] R J de Jesus ldquoAdaptive least square control in discrete time ofrobotic armsrdquo Soft Computing vol 19 no 12 pp 3665ndash36762015

[39] S Commuri S Jagannathan and F L Lewis ldquoCMAC neuralnetwork control of robot manipulatorsrdquo Journal of RoboticSystems vol 14 no 6 pp 465ndash482 1997

[40] J S Albus ldquoTheoretical and experimental aspects of a cerebellarmodelrdquoDevelopmental Disabilities Research Reviews vol 17 pp93ndash101 1972

[41] B Yang R Bao and H Han ldquoRobust hybrid control based onPD and novel CMAC with improved architecture and learningscheme for electric load simulatorrdquo IEEE Transactions onIndustrial Electronics vol 61 no 10 pp 5271ndash5279 2014

[42] S Jagannathan and F L Lewis ldquoMultilayer discrete-timeneural-net controller with guaranteed performancerdquo IEEETransactions on Neural Networks and Learning Systems vol 7no 1 pp 107ndash130 1996

[43] S S Ge and J Wang ldquoRobust adaptive neural control for a classof perturbed strict feedback nonlinear systemsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no6 pp 1409ndash1419 2002

[44] Y H Kim F L Lewis and C T Abdallah ldquoA dynamic recurrentneural-network-based adaptive observer for a class of nonlinearsystemsrdquo Automatica vol 33 no 8 pp 1539ndash1543 1997

[45] J-Q Huang and F L Lewis ldquoNeural-network predictive controlfor nonlinear dynamic systems with time-delayrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 14 no 2pp 377ndash389 2003

[46] S S Ge F Hong and T H Lee ldquoAdaptive neural control ofnonlinear time-delay systems with unknown virtual controlcoefficientsrdquo IEEE Transactions on Systems Man and Cybernet-ics Part B Cybernetics vol 34 no 1 pp 499ndash516 2004

[47] J Na X M Ren and H Huang ldquoTime-delay positive feedbackcontrol for nonlinear time-delay systems with neural networkcompensationrdquo Zidonghua XuebaoActa Automatica Sinica vol34 no 9 pp 1196ndash1202 2008

[48] J Na X Ren C Shang and Y Guo ldquoAdaptive neural networkpredictive control for nonlinear pure feedback systems withinput delayrdquo Journal of Process Control vol 22 no 1 pp 194ndash206 2012

[49] R R Selmic and F L Lewis ldquoNeural-network approximationof piecewise continuous functions application to friction com-pensationrdquo IEEE Transactions on Neural Networks and LearningSystems vol 13 no 3 pp 745ndash751 2002

[50] H Zhang and F L Lewis ldquoAdaptive cooperative trackingcontrol of higher-order nonlinear systems with unknowndynamicsrdquo Automatica vol 48 no 7 pp 1432ndash1439 2012

[51] J Na X Ren and D Zheng ldquoAdaptive control for nonlinearpure-feedback systems with high-order sliding mode observerrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 24 no 3 pp 370ndash382 2013

[52] J Na Q Chen X Ren and Y Guo ldquoAdaptive prescribedperformancemotion control of servomechanisms with frictioncompensationrdquo IEEE Transactions on Industrial Electronics vol61 no 1 pp 486ndash494 2014

[53] J Na X Ren G Herrmann and Z Qiao ldquoAdaptive neuraldynamic surface control for servo systems with unknown dead-zonerdquoControl Engineering Practice vol 19 no 11 pp 1328ndash13432011

[54] G Li J Na D P Stoten and X Ren ldquoAdaptive neural networkfeedforward control for dynamically substructured systemsrdquoIEEE Transactions on Control Systems Technology vol 22 no3 pp 944ndash954 2014

Complexity 11

[21] T Zhang S S Ge and C C Hang ldquoAdaptive neural networkcontrol for strict-feedback nonlinear systems using backstep-ping designrdquo Automatica vol 36 no 12 pp 1835ndash1846 2000

[22] S S Ge and T Zhang ldquoNeural-network control of nonaffinenonlinear system with zero dynamics by state and outputfeedbackrdquo IEEE Transactions on Neural Networks and LearningSystems vol 14 no 4 pp 900ndash918 2003

[23] S S Ge C Yang and T H Lee ldquoAdaptive predictive controlusing neural network for a class of pure-feedback systems in dis-crete timerdquo IEEE Transactions on Neural Networks and LearningSystems vol 19 no 9 pp 1599ndash1614 2008

[24] F W Lewis S Jagannathan and A Yesildirak Neural networkcontrol of robotmanipulators and non-linear systems CRCPress1998

[25] S Jagannathan and F L Lewis ldquoIdentification of nonlineardynamical systems using multilayered neural networksrdquo Auto-matica vol 32 no 12 pp 1707ndash1712 1996

[26] D Vrabie and F Lewis ldquoNeural network approach tocontinuous-time direct adaptive optimal control for partiallyunknown nonlinear systemsrdquo Neural Networks vol 22 no 3pp 237ndash246 2009

[27] C Yang S S Ge and T H Lee ldquoOutput feedback adaptive con-trol of a class of nonlinear discrete-time systems with unknowncontrol directionsrdquo Automatica vol 45 no 1 pp 270ndash2762009

[28] C Yang Z Li and J Li ldquoTrajectory planning and optimizedadaptive control for a class of wheeled inverted pendulumvehicle modelsrdquo IEEE Transactions on Cybernetics vol 43 no 1pp 24ndash36 2013

[29] Y Jiang C Yang and H Ma ldquoA review of fuzzy logic andneural network based intelligent control design for discrete-time systemsrdquo Discrete Dynamics in Nature and Society ArticleID 7217364 Art ID 7217364 11 pages 2016

[30] Y Jiang C Yang S-L Dai and B Ren ldquoDeterministic learningenhanced neutral network control of unmanned helicopterrdquoInternational Journal of Advanced Robotic Systems vol 13 no6 pp 1ndash12 2016

[31] Y Jiang Z Liu C Chen and Y Zhang ldquoAdaptive robust fuzzycontrol for dual arm robot with unknown input deadzonenonlinearityrdquo Nonlinear Dynamics vol 81 no 3 pp 1301ndash13142015

[32] httpsstemcellthailandorgneurons-definition-function-neurotransmitters

[33] MDefoort T Floquet A Kokosy andW Perruquetti ldquoSliding-mode formation control for cooperative autonomous mobilerobotsrdquo IEEE Transactions on Industrial Electronics vol 55 no11 pp 3944ndash3953 2008

[34] X Liu C Yang Z Chen MWang and C Su ldquoNeuro-adaptiveobserver based control of flexible joint robotrdquoNeurocomputing2017

[35] F Hamerlain T Floquet and W Perruquetti ldquoExperimentaltests of a slidingmode controller for trajectory tracking of a car-like mobile robotrdquo Robotica vol 32 no 1 pp 63ndash76 2014

[36] R J de Jesus ldquoDiscrete time control based in neural networksfor pendulumsrdquo Applied Soft Computing 2017

[37] Y Pan M J Er T Sun B Xu and H Yu ldquoAdaptive fuzzy PDcontrol with stable Hinfin tracking guaranteerdquo Neurocomputingvol 237 pp 71ndash78 2017

[38] R J de Jesus ldquoAdaptive least square control in discrete time ofrobotic armsrdquo Soft Computing vol 19 no 12 pp 3665ndash36762015

[39] S Commuri S Jagannathan and F L Lewis ldquoCMAC neuralnetwork control of robot manipulatorsrdquo Journal of RoboticSystems vol 14 no 6 pp 465ndash482 1997

[40] J S Albus ldquoTheoretical and experimental aspects of a cerebellarmodelrdquoDevelopmental Disabilities Research Reviews vol 17 pp93ndash101 1972

[41] B Yang R Bao and H Han ldquoRobust hybrid control based onPD and novel CMAC with improved architecture and learningscheme for electric load simulatorrdquo IEEE Transactions onIndustrial Electronics vol 61 no 10 pp 5271ndash5279 2014

[42] S Jagannathan and F L Lewis ldquoMultilayer discrete-timeneural-net controller with guaranteed performancerdquo IEEETransactions on Neural Networks and Learning Systems vol 7no 1 pp 107ndash130 1996

[43] S S Ge and J Wang ldquoRobust adaptive neural control for a classof perturbed strict feedback nonlinear systemsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 13 no6 pp 1409ndash1419 2002

[44] Y H Kim F L Lewis and C T Abdallah ldquoA dynamic recurrentneural-network-based adaptive observer for a class of nonlinearsystemsrdquo Automatica vol 33 no 8 pp 1539ndash1543 1997

[45] J-Q Huang and F L Lewis ldquoNeural-network predictive controlfor nonlinear dynamic systems with time-delayrdquo IEEE Transac-tions on Neural Networks and Learning Systems vol 14 no 2pp 377ndash389 2003

[46] S S Ge F Hong and T H Lee ldquoAdaptive neural control ofnonlinear time-delay systems with unknown virtual controlcoefficientsrdquo IEEE Transactions on Systems Man and Cybernet-ics Part B Cybernetics vol 34 no 1 pp 499ndash516 2004

[47] J Na X M Ren and H Huang ldquoTime-delay positive feedbackcontrol for nonlinear time-delay systems with neural networkcompensationrdquo Zidonghua XuebaoActa Automatica Sinica vol34 no 9 pp 1196ndash1202 2008

[48] J Na X Ren C Shang and Y Guo ldquoAdaptive neural networkpredictive control for nonlinear pure feedback systems withinput delayrdquo Journal of Process Control vol 22 no 1 pp 194ndash206 2012

[49] R R Selmic and F L Lewis ldquoNeural-network approximationof piecewise continuous functions application to friction com-pensationrdquo IEEE Transactions on Neural Networks and LearningSystems vol 13 no 3 pp 745ndash751 2002

[50] H Zhang and F L Lewis ldquoAdaptive cooperative trackingcontrol of higher-order nonlinear systems with unknowndynamicsrdquo Automatica vol 48 no 7 pp 1432ndash1439 2012

[51] J Na X Ren and D Zheng ldquoAdaptive control for nonlinearpure-feedback systems with high-order sliding mode observerrdquoIEEE Transactions on Neural Networks and Learning Systemsvol 24 no 3 pp 370ndash382 2013

[52] J Na Q Chen X Ren and Y Guo ldquoAdaptive prescribedperformancemotion control of servomechanisms with frictioncompensationrdquo IEEE Transactions on Industrial Electronics vol61 no 1 pp 486ndash494 2014

[53] J Na X Ren G Herrmann and Z Qiao ldquoAdaptive neuraldynamic surface control for servo systems with unknown dead-zonerdquoControl Engineering Practice vol 19 no 11 pp 1328ndash13432011

[54] G Li J Na D P Stoten and X Ren ldquoAdaptive neural networkfeedforward control for dynamically substructured systemsrdquoIEEE Transactions on Control Systems Technology vol 22 no3 pp 944ndash954 2014

12 Complexity

[55] B Xu Z Shi C Yang and F Sun ldquoComposite neural dynamicsurface control of a class of uncertain nonlinear systems instrict-feedback formrdquo IEEE Transactions on Cybernetics 2014

[56] M Chen and S Ge ldquoAdaptive neural output feedback controlof uncertain nonlinear systems with unknown hysteresis usingdisturbance observerrdquo IEEE Transactions on Industrial Electron-ics 2015

[57] M Chen and S S Ge ldquoDirect adaptive neural control for a classof uncertain nonaffine nonlinear systems based on disturbanceobserverrdquo IEEE Transactions on Cybernetics vol 43 no 4 pp1213ndash1225 2013

[58] M Chen G Tao and B Jiang ldquoDynamic surface control usingneural networks for a class of uncertain nonlinear systems withinput saturationrdquo IEEE Transactions on Neural Networks andLearning Systems vol 26 no 9 pp 2086ndash2097 2015

[59] P J Werbos ldquoApproximate dynamic programming for real-time control and neural modelingrdquo in Handbook of IntelligentControl 1992

[60] F-Y Wang H Zhang and D Liu ldquoAdaptive dynamic pro-gramming an introductionrdquo IEEE Computational IntelligenceMagazine vol 4 no 2 pp 39ndash47 2009

[61] F L Lewis D Vrabie and K Vamvoudakis ldquoReinforcementlearning and feedback control using natural decision methodsto design optimal adaptive controllersrdquo IEEE Control SystemsMagazine vol 32 no 6 pp 76ndash105 2012

[62] F L Lewis andD Vrabie ldquoReinforcement learning and adaptivedynamic programming for feedback controlrdquo IEEE Circuits andSystems Magazine vol 9 no 3 pp 32ndash50 2009

[63] A Al-Tamimi F L Lewis and M Abu-Khalaf ldquoDiscrete-timenonlinear HJB solution using approximate dynamic program-ming convergence proofrdquo IEEE Transactions on Systems Manand Cybernetics Part B Cybernetics vol 38 no 4 pp 943ndash9492008

[64] H Zhang Y Luo and D Liu ldquoNeural-network-based near-optimal control for a class of discrete-time affine nonlinearsystems with control constraintsrdquo IEEE Transactions on NeuralNetworks and Learning Systems vol 20 no 9 pp 1490ndash15032009

[65] D Wang D Liu Q Wei D Zhao and N Jin ldquoOptimal controlof unknown nonaffine nonlinear discrete-time systems basedon adaptive dynamic programmingrdquo Automatica vol 48 no 8pp 1825ndash1832 2012

[66] H He Z Ni and J Fu ldquoA three-network architecture for on-line learning and optimization based on adaptive dynamic pro-grammingrdquo Neurocomputing vol 78 no 1 pp 3ndash13 2012

[67] D Liu and Q Wei ldquoPolicy iteration adaptive dynamic pro-gramming algorithm for discrete-time nonlinear systemsrdquo IEEETransactions on Neural Networks and Learning Systems vol 25no 3 pp 621ndash634 2014

[68] D Liu X Yang D Wang and Q Wei ldquoReinforcement-learning-based robust controller design for continuous-timeuncertain nonlinear systems subject to input constraintsrdquo IEEETransactions on Cybernetics vol 45 no 7 pp 1372ndash1385 2015

[69] J Fu H He and X Zhou ldquoAdaptive learning and control forMIMO systembased on adaptive dynamic programmingrdquo IEEETransactions on Neural Networks and Learning Systems vol 22no 7 pp 1133ndash1148 2011

[70] Y Lv J Na Q Yang X Wu and Y Guo ldquoOnline adaptiveoptimal control for continuous-time nonlinear systems withcompletely unknown dynamicsrdquo International Journal of Con-trol vol 89 no 1 pp 99ndash112 2016

[71] J Na and G Herrmann ldquoOnline adaptive approximate optimaltracking control with simplified dual approximation structurefor continuous-time unknown nonlinear systemsrdquo IEEECAAJournal of Automatica Sinica vol 1 no 4 pp 412ndash422 2014

[72] X Yao ldquoEvolving artificial neural networksrdquo Proceedings of theIEEE vol 87 no 9 pp 1423ndash1447 1999

[73] S F Ding H Li C Y Su J Z Yu and F X Jin ldquoEvolution-ary artificial neural networks a reviewrdquo Artificial IntelligenceReview vol 39 no 3 pp 251ndash260 2013

[74] X Yao and Y Liu ldquoA new evolutionary system for evolving arti-ficial neural networksrdquo IEEE Transactions on Neural Networksand Learning Systems vol 8 no 3 pp 694ndash713 1997

[75] Y Li and A Hauszligler ldquoArtificial evolution of neural networksand its application to feedback controlrdquo Artificial Intelligence inEngineering vol 10 no 2 pp 143ndash152 1996

[76] C-K Goh E-J Teoh and K C Tan ldquoHybrid multiobjectiveevolutionary design for artificial neural networksrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 19 no 9pp 1531ndash1548 2008

[77] K C Tan and Y Li ldquoGrey-box model identification via evolu-tionary computingrdquo Control Engineering Practice vol 10 no 7pp 673ndash684 2002

[78] L Wang C K Tan and C M Chew Evolutionary roboticsfrom algorithms to implementations Chew C M Evolutionaryrobotics from algorithms to implementations[M] NJ 2006

[79] J Zhang Z-H Zhang Y Lin et al ldquoEvolutionary computationmeets machine learning a surveyrdquo IEEE Computational Intelli-gence Magazine vol 6 no 4 pp 68ndash75 2011

[80] C Yang X Wang L Cheng and H Ma ldquoNeural-learning-based telerobot control with guaranteed performancerdquo IEEETransactions on Cybernetics 2017

[81] C Yang K Huang H Cheng Y Li and C Su ldquoHaptic identifi-cation by ELM-controlled uncertain manipulatorrdquo IEEE Trans-actions on Systems Man and Cybernetics Systems vol 47 no8 2017

[82] C Yang Z Li R Cui and B Xu ldquoNeural network-basedmotion control of an underactuated wheeled inverted pen-dulum modelrdquo IEEE Transactions on Neural Networks andLearning Systems 2014

[83] C Yang T Teng B Xu Z Li J Na and C Su ldquoGlobal adaptivetracking control of robot manipulators using neural networkswith finite-time learning convergencerdquo International Journal ofControl Automation and Systems vol 15 no 4 pp 1916ndash19242017

[84] C Yang Y Jiang Z Li W He and C-Y Su ldquoNeural controlof bimanual robots with guaranteed global stability and motionprecisionrdquo IEEE Transactions on Industrial Informatics 2017

[85] R Cui and W Yan ldquoMutual synchronization of multiple robotmanipulators with unknown dynamicsrdquo Journal of Intelligent ampRobotic Systems vol 68 no 2 pp 105ndash119 2012

[86] L Cheng Z-G Hou M Tan and W J Zhang ldquoTrackingcontrol of a closed-chain five-bar robotwith two degrees of free-dom by integration of an approximation-based approach andmechanical designrdquo IEEE Transactions on Systems Man andCybernetics Part B Cybernetics vol 42 no 5 pp 1470ndash14792012

[87] C Yang XWang Z Li Y Li and C Su ldquoTeleoperation controlbased on combination of wave variable and neural networksrdquoIEEE Transactions on Systems Man and Cybernetics Systems2017

Complexity 13

[88] C Yang J Luo Y Pan Z Liu and C Su ldquoPersonalized variablegain control with tremor attenuation for robot teleoperationrdquoIEEE Transactions on Systems Man and Cybernetics Systemspp 1ndash12 2017

[89] L Cheng Z-G Hou and M Tan ldquoAdaptive neural networktracking control for manipulators with uncertain kinematicsdynamics and actuator modelrdquo Automatica vol 45 no 10 pp2312ndash2318 2009

[90] W He Y Dong and C Sun ldquoAdaptive Neural ImpedanceControl of a Robotic Manipulator with Input Saturationrdquo IEEETransactions on Systems Man and Cybernetics Systems vol 46no 3 pp 334ndash344 2016

[91] WHe A O David Z Yin and C Sun ldquoNeural network controlof a robotic manipulator with input deadzone and outputconstraintrdquo IEEE Transactions on Systems Man and Cybernet-ics Systems 2015

[92] W He Z Yin and C Sun ldquoAdaptive Neural Network Controlof a Marine Vessel With Constraints Using the AsymmetricBarrier Lyapunov Functionrdquo IEEE Transactions on Cybernetics2016

[93] W He Y Chen and Z Yin ldquoAdaptive neural network controlof an uncertain robot with full-state constraintsrdquo IEEE Transac-tions on Cybernetics vol 46 no 3 pp 620ndash629 2016

[94] C Sun W He and J Hong ldquoNeural Network Control of aFlexible Robotic Manipulator Using the Lumped Spring-MassModelrdquo IEEE Transactions on Systems Man and CyberneticsSystems vol 47 no 8 pp 1863ndash1874 2017

[95] W He Y Ouyang and J Hong ldquoVibration Control of a FlexibleRobotic Manipulator in the Presence of Input Deadzonerdquo IEEETransactions on Industrial Informatics vol 13 no 1 pp 48ndash592017

[96] R Cui X Zhang and D Cui ldquoAdaptive sliding-mode attitudecontrol for autonomous underwater vehicles with input nonlin-earitiesrdquo Ocean Engineering vol 123 pp 45ndash54 2016

[97] R Cui C Yang Y Li and S Sharma ldquoAdaptive Neural NetworkControl of AUVs With Control Input Nonlinearities UsingReinforcement Learningrdquo IEEE Transactions on Systems Manand Cybernetics Systems vol 47 no 6 pp 1019ndash1029 2017

[98] B Xu D Wang Y Zhang and Z Shi ldquoDOB based neuralcontrol of flexible hypersonic flight vehicle considering windeffectsrdquo IEEE Transactions on Industrial Electronics vol PP no99 p 1 2017

[99] B Xu C Yang and Y Pan ldquoGlobal neural dynamic surfacetracking control of strict-feedback systems with application tohypersonic flight vehiclerdquo IEEE Transactions on Neural Net-works and Learning Systems vol 26 no 10 pp 2563ndash2575 2015

[100] Y Li S S Ge and C Yang ldquoLearning impedance control forphysical robot-environment interactionrdquo International Journalof Control vol 85 no 2 pp 182ndash193 2012

[101] Y Li S S Ge Q Zhang and T Lee ldquoNeural networksimpedance control of robots interacting with environmentsrdquoIET Control Theory amp Applications vol 7 no 11 pp 1509ndash15192013

[102] Y Li and S S Ge ldquoImpedance learning for robots interactingwith unknown environmentsrdquo IEEE Transactions on ControlSystems Technology vol 22 no 4 pp 1422ndash1432 2014

[103] C Wang Y Li S S Ge and T Lee ldquoOptimal critic learningfor robot control in time-varying environmentsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 26 no10 pp 2301ndash2310 2015

[104] Y Li L Chen K P Tee and Q Li ldquoReinforcement learningcontrol for coordinated manipulation of multi-robotsrdquo Neuro-computing vol 170 pp 168ndash175 2015

[105] Y Li and S S Ge ldquoHumanmdashrobot collaboration based onmotion intention estimationrdquo IEEEASME Transactions onMechatronics vol 19 no 3 pp 1007ndash1014 2013

[106] Y Li K P Tee W L Chan R Yan Y Chua and D KLimbu ldquoContinuous RoleAdaptation forHuman-Robot SharedControlrdquo IEEE Transactions on Robotics vol 31 no 3 pp 672ndash681 2015

[107] Y Li K P Tee R Yan W L Chan and YWu ldquoA Framework ofHuman-RobotCoordinationBased onGameTheory andPolicyIterationrdquo IEEE Transactions on Robotics vol 32 no 6 pp1408ndash1418 2016

[108] A Clark Surfing uncertainty Prediction action and the embod-ied mind Oxford University Press 2015

[109] J Zhong C Weber and S Wermter ldquoLearning features andpredictive transformation encoding based on a horizontal prod-uct modelrdquo Neural Networks and Machine LearningICANN pp539ndash546 2012

[110] J Zhong C Weber and S Wermter ldquoA predictive networkarchitecture for a robust and smooth robot docking behaviorrdquoJournal of Behavioral Robotics vol 3 no 4 pp 172ndash180 2012

[111] W Prinz ldquoPerception and Action Planningrdquo European Journalof Cognitive Psychology vol 9 no 2 pp 129ndash154 1997

[112] J Zhong M Peniak J Tani T Ogata and A CangelosildquoSensorimotor Input as a Language Generalisation Tool ANeurorobotics Model for Generation and Generalisation ofNoun-Verb Combinations with Sensorimotor Inputsrdquo TechRep arXiv 160503261 2016 arXiv160503261

[113] H T Siegelmann and E D Sontag ldquoOn the computationalpower of neural netsrdquo Journal of Computer and System Sciencesvol 50 no 1 pp 132ndash150 1995

[114] J Zhong Artificial Neural Models for Feedback Pathways forSensorimotor Integration

[115] J Tani M Ito and Y Sugita ldquoSelf-organization of distributedlyrepresented multiple behavior schemata in a mirror systemReviews of robot experiments using RNNPBrdquoNeural Networksvol 17 no 8-9 pp 1273ndash1289 2004

[116] W Hinoshita H Arie J Tani H G Okuno and T OgataldquoEmergence of hierarchical structure mirroring linguistic com-position in a recurrent neural networkrdquo Neural Networks vol24 no 4 pp 311ndash320 2011

[117] J Tani Exploring robotic minds actions symbols and conscious-ness as self-organizing dynamic phenomena Oxford UniversityPress 2016

[118] A Ahmadi and J Tani ldquoHow can a recurrent neurodynamicpredictive coding model cope with fluctuation in temporalpatterns Robotic experiments on imitative interactionrdquoNeuralNetworks vol 92 pp 3ndash16 2017

[119] J Zhong A Cangelosi and S Wermter ldquoToward a self-organizing pre-symbolic neural model representing sensori-motor primitivesrdquo Frontiers in Behavioral Neuroscience vol 8article no 22 2014

[120] H K Abbas R M Zablotowicz and H A Bruns ldquoModelingthe colonization of maize by toxigenic and non-toxigenicAspergillus flavus strains implications for biological controlrdquoWorld Mycotoxin Journal vol 1 no 3 pp 333ndash340 2008

[121] J Zhong and L Canamero ldquoFrom continuous affective spaceto continuous expression space Non-verbal behaviour recog-nition and generationrdquo in Proceedings of the 4th Joint IEEE

14 Complexity

International Conference on Development and Learning and onEpigenetic Robotics IEEE ICDL-EPIROB 2014 pp 75ndash80 itaOctober 2014

[122] J Zhong A Cangelosi and T Ogata ldquoToward Abstractionfrom Multi-modal Data Empirical Studies on Multiple Time-scale RecurrentModelsrdquo inProceedings of the International JointConference on Artificial Neural Networks (IJCNN) 2017

[123] G Park and J Tani ldquoDevelopment of compositional and con-textual communicable congruence in robots by using dynamicneural network modelsrdquo Neural Networks vol 72 pp 109ndash1222015

[124] A Ahmadi and J Tani ldquoBridging the gap between probabilisticand deterministic models a simulation study on a variationalBayes predictive coding recurrent neural network modelrdquo 2017httpsarxivorgabs170610240

[125] Y LeCun Y Bengio and G Hinton ldquoDeep learningrdquo Naturevol 521 no 7553 pp 436ndash444 2015

[126] J Schmidhuber ldquoDeep learning in neural networks anoverviewrdquo Neural Networks vol 61 pp 85ndash117 2015

[127] A Krizhevsky I Sutskever andG EHinton ldquoImagenet classifi-cation with deep convolutional neural networksrdquo in Proceedingsof the 26th Annual Conference on Neural Information ProcessingSystems (NIPS rsquo12) pp 1097ndash1105 Lake Tahoe Nev USADecember 2012

[128] W Liu Z Wang X Liu N Zeng Y Liu and F E Alsaadi ldquoAsurvey of deep neural network architectures and their applica-tionsrdquo Neurocomputing vol 234 pp 11ndash26 2017

[129] G E Hinton and R R Salakhutdinov ldquoReducing the dimen-sionality of data with neural networksrdquo American Associationfor the Advancement of Science Science vol 313 no 5786 pp504ndash507 2006

[130] B C Pijanowski A Tayyebi J Doucette B K Pekin DBraun and J Plourde ldquoA big data urban growth simulation at anational scale configuring the GIS and neural network basedLand Transformation Model to run in a High PerformanceComputing (HPC) environmentrdquo Environmental Modelling ampSoftware vol 51 pp 250ndash268 2014

[131] D C Ciresan UMeier LMGambardella and J SchmidhuberldquoDeep big simple neural nets for handwritten digit recogni-tionrdquo Neural Computation vol 22 no 12 pp 3207ndash3220 2010

Complexity 13

[88] C Yang J Luo Y Pan Z Liu and C Su ldquoPersonalized variablegain control with tremor attenuation for robot teleoperationrdquoIEEE Transactions on Systems Man and Cybernetics Systemspp 1ndash12 2017

[89] L Cheng Z-G Hou and M Tan ldquoAdaptive neural networktracking control for manipulators with uncertain kinematicsdynamics and actuator modelrdquo Automatica vol 45 no 10 pp2312ndash2318 2009

[90] W He Y Dong and C Sun ldquoAdaptive Neural ImpedanceControl of a Robotic Manipulator with Input Saturationrdquo IEEETransactions on Systems Man and Cybernetics Systems vol 46no 3 pp 334ndash344 2016

[91] WHe A O David Z Yin and C Sun ldquoNeural network controlof a robotic manipulator with input deadzone and outputconstraintrdquo IEEE Transactions on Systems Man and Cybernet-ics Systems 2015

[92] W He Z Yin and C Sun ldquoAdaptive Neural Network Controlof a Marine Vessel With Constraints Using the AsymmetricBarrier Lyapunov Functionrdquo IEEE Transactions on Cybernetics2016

[93] W He Y Chen and Z Yin ldquoAdaptive neural network controlof an uncertain robot with full-state constraintsrdquo IEEE Transac-tions on Cybernetics vol 46 no 3 pp 620ndash629 2016

[94] C Sun W He and J Hong ldquoNeural Network Control of aFlexible Robotic Manipulator Using the Lumped Spring-MassModelrdquo IEEE Transactions on Systems Man and CyberneticsSystems vol 47 no 8 pp 1863ndash1874 2017

[95] W He Y Ouyang and J Hong ldquoVibration Control of a FlexibleRobotic Manipulator in the Presence of Input Deadzonerdquo IEEETransactions on Industrial Informatics vol 13 no 1 pp 48ndash592017

[96] R Cui X Zhang and D Cui ldquoAdaptive sliding-mode attitudecontrol for autonomous underwater vehicles with input nonlin-earitiesrdquo Ocean Engineering vol 123 pp 45ndash54 2016

[97] R Cui C Yang Y Li and S Sharma ldquoAdaptive Neural NetworkControl of AUVs With Control Input Nonlinearities UsingReinforcement Learningrdquo IEEE Transactions on Systems Manand Cybernetics Systems vol 47 no 6 pp 1019ndash1029 2017

[98] B Xu D Wang Y Zhang and Z Shi ldquoDOB based neuralcontrol of flexible hypersonic flight vehicle considering windeffectsrdquo IEEE Transactions on Industrial Electronics vol PP no99 p 1 2017

[99] B Xu C Yang and Y Pan ldquoGlobal neural dynamic surfacetracking control of strict-feedback systems with application tohypersonic flight vehiclerdquo IEEE Transactions on Neural Net-works and Learning Systems vol 26 no 10 pp 2563ndash2575 2015

[100] Y Li S S Ge and C Yang ldquoLearning impedance control forphysical robot-environment interactionrdquo International Journalof Control vol 85 no 2 pp 182ndash193 2012

[101] Y Li S S Ge Q Zhang and T Lee ldquoNeural networksimpedance control of robots interacting with environmentsrdquoIET Control Theory amp Applications vol 7 no 11 pp 1509ndash15192013

[102] Y Li and S S Ge ldquoImpedance learning for robots interactingwith unknown environmentsrdquo IEEE Transactions on ControlSystems Technology vol 22 no 4 pp 1422ndash1432 2014

[103] C Wang Y Li S S Ge and T Lee ldquoOptimal critic learningfor robot control in time-varying environmentsrdquo IEEE Trans-actions on Neural Networks and Learning Systems vol 26 no10 pp 2301ndash2310 2015

[104] Y Li L Chen K P Tee and Q Li ldquoReinforcement learningcontrol for coordinated manipulation of multi-robotsrdquo Neuro-computing vol 170 pp 168ndash175 2015

[105] Y Li and S S Ge ldquoHumanmdashrobot collaboration based onmotion intention estimationrdquo IEEEASME Transactions onMechatronics vol 19 no 3 pp 1007ndash1014 2013

[106] Y Li K P Tee W L Chan R Yan Y Chua and D KLimbu ldquoContinuous RoleAdaptation forHuman-Robot SharedControlrdquo IEEE Transactions on Robotics vol 31 no 3 pp 672ndash681 2015

[107] Y Li K P Tee R Yan W L Chan and YWu ldquoA Framework ofHuman-RobotCoordinationBased onGameTheory andPolicyIterationrdquo IEEE Transactions on Robotics vol 32 no 6 pp1408ndash1418 2016

[108] A Clark Surfing uncertainty Prediction action and the embod-ied mind Oxford University Press 2015

[109] J Zhong C Weber and S Wermter ldquoLearning features andpredictive transformation encoding based on a horizontal prod-uct modelrdquo Neural Networks and Machine LearningICANN pp539ndash546 2012

[110] J Zhong C Weber and S Wermter ldquoA predictive networkarchitecture for a robust and smooth robot docking behaviorrdquoJournal of Behavioral Robotics vol 3 no 4 pp 172ndash180 2012

[111] W Prinz ldquoPerception and Action Planningrdquo European Journalof Cognitive Psychology vol 9 no 2 pp 129ndash154 1997

[112] J Zhong M Peniak J Tani T Ogata and A CangelosildquoSensorimotor Input as a Language Generalisation Tool ANeurorobotics Model for Generation and Generalisation ofNoun-Verb Combinations with Sensorimotor Inputsrdquo TechRep arXiv 160503261 2016 arXiv160503261

[113] H T Siegelmann and E D Sontag ldquoOn the computationalpower of neural netsrdquo Journal of Computer and System Sciencesvol 50 no 1 pp 132ndash150 1995

[114] J Zhong Artificial Neural Models for Feedback Pathways forSensorimotor Integration

[115] J Tani M Ito and Y Sugita ldquoSelf-organization of distributedlyrepresented multiple behavior schemata in a mirror systemReviews of robot experiments using RNNPBrdquoNeural Networksvol 17 no 8-9 pp 1273ndash1289 2004

[116] W Hinoshita H Arie J Tani H G Okuno and T OgataldquoEmergence of hierarchical structure mirroring linguistic com-position in a recurrent neural networkrdquo Neural Networks vol24 no 4 pp 311ndash320 2011

[117] J Tani Exploring robotic minds actions symbols and conscious-ness as self-organizing dynamic phenomena Oxford UniversityPress 2016

[118] A Ahmadi and J Tani ldquoHow can a recurrent neurodynamicpredictive coding model cope with fluctuation in temporalpatterns Robotic experiments on imitative interactionrdquoNeuralNetworks vol 92 pp 3ndash16 2017

[119] J Zhong A Cangelosi and S Wermter ldquoToward a self-organizing pre-symbolic neural model representing sensori-motor primitivesrdquo Frontiers in Behavioral Neuroscience vol 8article no 22 2014

[120] H K Abbas R M Zablotowicz and H A Bruns ldquoModelingthe colonization of maize by toxigenic and non-toxigenicAspergillus flavus strains implications for biological controlrdquoWorld Mycotoxin Journal vol 1 no 3 pp 333ndash340 2008

[121] J Zhong and L Canamero ldquoFrom continuous affective spaceto continuous expression space Non-verbal behaviour recog-nition and generationrdquo in Proceedings of the 4th Joint IEEE

14 Complexity

International Conference on Development and Learning and onEpigenetic Robotics IEEE ICDL-EPIROB 2014 pp 75ndash80 itaOctober 2014

[122] J Zhong A Cangelosi and T Ogata ldquoToward Abstractionfrom Multi-modal Data Empirical Studies on Multiple Time-scale RecurrentModelsrdquo inProceedings of the International JointConference on Artificial Neural Networks (IJCNN) 2017

[123] G Park and J Tani ldquoDevelopment of compositional and con-textual communicable congruence in robots by using dynamicneural network modelsrdquo Neural Networks vol 72 pp 109ndash1222015

[124] A Ahmadi and J Tani ldquoBridging the gap between probabilisticand deterministic models a simulation study on a variationalBayes predictive coding recurrent neural network modelrdquo 2017httpsarxivorgabs170610240

[125] Y LeCun Y Bengio and G Hinton ldquoDeep learningrdquo Naturevol 521 no 7553 pp 436ndash444 2015

[126] J Schmidhuber ldquoDeep learning in neural networks anoverviewrdquo Neural Networks vol 61 pp 85ndash117 2015

[127] A Krizhevsky I Sutskever andG EHinton ldquoImagenet classifi-cation with deep convolutional neural networksrdquo in Proceedingsof the 26th Annual Conference on Neural Information ProcessingSystems (NIPS rsquo12) pp 1097ndash1105 Lake Tahoe Nev USADecember 2012

[128] W Liu Z Wang X Liu N Zeng Y Liu and F E Alsaadi ldquoAsurvey of deep neural network architectures and their applica-tionsrdquo Neurocomputing vol 234 pp 11ndash26 2017

[129] G E Hinton and R R Salakhutdinov ldquoReducing the dimen-sionality of data with neural networksrdquo American Associationfor the Advancement of Science Science vol 313 no 5786 pp504ndash507 2006

[130] B C Pijanowski A Tayyebi J Doucette B K Pekin DBraun and J Plourde ldquoA big data urban growth simulation at anational scale configuring the GIS and neural network basedLand Transformation Model to run in a High PerformanceComputing (HPC) environmentrdquo Environmental Modelling ampSoftware vol 51 pp 250ndash268 2014

[131] D C Ciresan UMeier LMGambardella and J SchmidhuberldquoDeep big simple neural nets for handwritten digit recogni-tionrdquo Neural Computation vol 22 no 12 pp 3207ndash3220 2010

14 Complexity

International Conference on Development and Learning and onEpigenetic Robotics IEEE ICDL-EPIROB 2014 pp 75ndash80 itaOctober 2014

[122] J Zhong A Cangelosi and T Ogata ldquoToward Abstractionfrom Multi-modal Data Empirical Studies on Multiple Time-scale RecurrentModelsrdquo inProceedings of the International JointConference on Artificial Neural Networks (IJCNN) 2017

[123] G Park and J Tani ldquoDevelopment of compositional and con-textual communicable congruence in robots by using dynamicneural network modelsrdquo Neural Networks vol 72 pp 109ndash1222015

[124] A Ahmadi and J Tani ldquoBridging the gap between probabilisticand deterministic models a simulation study on a variationalBayes predictive coding recurrent neural network modelrdquo 2017httpsarxivorgabs170610240

[125] Y LeCun Y Bengio and G Hinton ldquoDeep learningrdquo Naturevol 521 no 7553 pp 436ndash444 2015

[126] J Schmidhuber ldquoDeep learning in neural networks anoverviewrdquo Neural Networks vol 61 pp 85ndash117 2015

[127] A Krizhevsky I Sutskever andG EHinton ldquoImagenet classifi-cation with deep convolutional neural networksrdquo in Proceedingsof the 26th Annual Conference on Neural Information ProcessingSystems (NIPS rsquo12) pp 1097ndash1105 Lake Tahoe Nev USADecember 2012

[128] W Liu Z Wang X Liu N Zeng Y Liu and F E Alsaadi ldquoAsurvey of deep neural network architectures and their applica-tionsrdquo Neurocomputing vol 234 pp 11ndash26 2017

[129] G E Hinton and R R Salakhutdinov ldquoReducing the dimen-sionality of data with neural networksrdquo American Associationfor the Advancement of Science Science vol 313 no 5786 pp504ndash507 2006

[130] B C Pijanowski A Tayyebi J Doucette B K Pekin DBraun and J Plourde ldquoA big data urban growth simulation at anational scale configuring the GIS and neural network basedLand Transformation Model to run in a High PerformanceComputing (HPC) environmentrdquo Environmental Modelling ampSoftware vol 51 pp 250ndash268 2014

[131] D C Ciresan UMeier LMGambardella and J SchmidhuberldquoDeep big simple neural nets for handwritten digit recogni-tionrdquo Neural Computation vol 22 no 12 pp 3207ndash3220 2010

Submit your manuscripts athttpswwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of