Intelligent Control

Module III- Neural ControlLecture 1

Neural Control - A Review

Laxmidhar Behera

Department of Electrical EngineeringIndian Institute of Technology, Kanpur

. – p.1/46

Topics to be Covered

Adaptive control paradigms:Direct adaptive controlIndirect adaptive control

Nonlinear systems - Important classes

Neural network models

Other neural control architectures

Summary

. – p.2/46

Neural Control: A broad classification

Indirect Adaptive Control

Direct Adaptive Control

. – p.3/46

Direct Adaptive control

ym

ec

u

PlantController

Referencemodel

r

+

-yp

ec

The parameters of controller are directly adjusted to

reduce some norm of the output error.

. – p.4/46

Direct Adaptive Control: Important Works

Gaussian networks for direct adaptive control. R. M. Sanner andJ. J. E. Slotine, IEEE TNN November 1992.

Multilayer discrete-time neural net controller with guaranteedperformance. S. Jagannathan and F. L. Lewis, IEEE TNNJanuary 1996.

A Direct adaptive neural-network control for unknown nonlinearsystems and its application, J. R. Noriega and Hong Wang, IEEETNN, January 1998.

nonaffine system

Fluid-flow control

Robust neural-network control of rigid-link electrically driven

robots, C. Kwan, F. L. Lewis and M. Dawson, IEEE TNN July

1998.

. – p.5/46

Direct Adaptive control: Important works

Control of a nonholonomic mobile robot using neural networks,R. Fierro and F. L. Lewis, IEEE TNN, July 1998.

Robust Backstepping control of Induction motors using NeuralNetworks, C. M. Kwan and F. L. Lewis, IEEE TNN, September2000.

Control of a class of nonlinear Discrete-time systems usingmultilayer neural network, S. Jagannathan, IEEE, TNN,September 2001.

system nonlinearities don’t satisfy matching conditions

. – p.6/46

Direct adaptive control: important works

Adaptive output feedback control of Uncertain nonlinear systemsusing single hidden layer neural networks, Naira Hovakimyan,Flavio Nardi, Anthony Calise and Nakwan Kim, IEEE, TNN,November 2002.

Dynamics and dimension of regulated system may be unknown, but therelative degree of regulated output must be known.

Vander-pol oscillator and R-50 Helicopter.

Neural network control of nonaffine nonlinear system with zerodynamics by state and output feedback, S. S. Ge and J. Zhang,IEEE TNNN July 2003.

Temperature control of a thermal reactor.

. – p.7/46

Direct Adaptive Control: Important Works

Stable Adaptive Neurocontrol for Nonlinear Discrete-timesystems, Quanmin Zhu and Lingzhong Guo, IEEE, TNN, May2004.

Non-affine Discrete-time system

Recurrent Neural Network

Network weight update is derived from Lyapunov analysis

Connection between weight convergence and reconstruction error of thenetwork is established

Liquid-level system and Vanderpol oscillator

A stable NN based observer with application to Flexible-jointmanipulators, F. Abdollahi, H. A. Talebi and Rajnikant V. Patel,IEEE, TNN, January 2006.

Non-affine system

An observer that uses Nonlinear-in-parameters NN

No SPR assumption imposed

Lyapunov-stability analysis. – p.8/46

Indirect adaptive control

PlantController

pIdentification

model

modelReference

yp

ec

+

-

-

+

r u

ei

ei

yp

ym

The parameters of plant are first estimated and thecontroller is designed assuming that estimated plantparameters represent true plant parameters. Indirectadaptive control involves explicit system identification.

. – p.9/46

Indirect Adaptive Control: Important Works

Iterative inversion of neural networks and its application toadaptive control. D. A. Hoskins, J. N. Hwang and J. Vagners,IEEE TNN March 1992.

Neuro-adaptive hybrid controller for robot-manipulator trackingcontrol. L. Behera, S. Chaudhury and M. Gopal, IEE Proc.Control theory and applications. May 1996.

Indirect control of a class of nonlinear dynamic system. S. Mistry,S. L. Chang and Satish S. Nair, IEEE TNN July 1996.

On adaptive trajectory tracking of a robot manipulator usinginversion of its neural emulator. L. Behera, M. Gopal and S.Choudhury, IEEE TNN November 1996.

Indirect adaptive control via parallel dynamic neural networks. W.

Yu, A. S. Poznyak, IEE proc. on control theory and applications,

january 1999.. – p.10/46

Indirect adaptive control: Important works

Query based model learning and stable tracking of a robot armusing radial basis function network. Laxmidhar Behera,Computers and Electrical Engineering, Pergamon, 2003.

Adaptive inverse control of linear and nonlinear systems usingdynamic neural networks. G. L. Plett, IEEE TNN march 2003.

. – p.11/46

Nonlinear Systems: Some Important Classes

Affine systemStrict-feedback formSingularly-perturbed form

Non-affine system

Interconnected systems

. – p.12/46

Affine system

A nonlinear system in affine form is represented as

x = f(x) + g(x)u

Neural networks can be used under following two classes:

f(.) is unknown and g(.) is known.

f(.) and g(.) are unknown.

Define e = xd − x. This gives e = xd − f − gu. Choosing a controller

u =1

g(−f + xd + Ke)

we get closed loop error dynamics as e + Ke = f − f = f . Neural

network weight update law is chosen to keep the weights bounded.. – p.13/46

Affine system - Important works

Identification and control of dynamical systems using neuralnetworks, Narendra, K. S. and Parthasarathy, K., IEEE TNN,March 1990.

Adaptive control of a class of nonlinear discrete-time systemsusing neural networks. F. C. Chen and H. K. Khalil, IEEEAutomatic control, May 1995.

A neural approach for control of nonlinear systems with feedbacklinearization, Shouling He, Konrad Reif and Rolf Unbehauen,IEEE TNN, November 1998.

An adaptive tracking controller using neural networks for a classof nonlinear systems, Man Zhihong, H. R. Wu and M.Paliniswami, IEEE, TNN September 1998.

. – p.14/46

Strict feedback form

Consider the system of following form

x1 = F1(x1) + G1(x1)x2

x2 = F2(x1, x2) + G2(x1, x2)x3

x3 = F3(x1, x2, x3) + G3(x1, x2, x3)x4

. . . = . . .

xm = Fm(x1, x2, . . . , xm) + Gm(x1, x2, . . . , xm)u

where Fi and Gi are nonlinear functions that contain parametric and non-parametricuncertainties. Gi are assumed to be known and invertible.Control steps: Backstepping

Choose x2 = x2d so that x1 tracks x1d.

Choose x3 = x3d so that x2 tracks x2d and so on.

Finally, choose u so that xm tracks xmd.

Fi are approximated using neural networks and weights are updated tokeep the approximation error bounded.

. – p.15/46

Backstepping and relevant works

Robust backstepping control of nonlinear systems using neuralnetworks. C. Kwan and F. L. Lewis, IEEE SMC Part A, November2000.

Stable neural controller design for unknown nonlinear systemsusing backstepping. Y. Zhang, P. Y. Peng and Z. P. Jiang, IEEETNN, November 2000.

Adaptive observer backstepping control using neural networks.Y. C. Jin and J. A. Farrel, IEEE TNN, September 2001.

Nonlinear Adaptive Flight Control Using Backstepping andNeural Networks Controller. T. Lee and Y. Kim. Journal ofGuidance, control and dynamics, 2001.

Adaptive backstepping control using recurrent neural network forlinear induction motor drive. F. Lin, R. Wai, W. chou, S. Hsu,IEEE Trans. Industrial electronics, February 2002. . – p.16/46

Singularly perturbed model

A large class of nonlinear systems can be described by the equations

x1 = f1(x, u)

εx2 = f21(x1) + f22(x1)x2 + g2(x1)u

ε << 1, indicating that the dynamics of x2 are much faster thanthose of x1.

Slow/Fast subsystem decomposition

x1 = x1 + x1, x2 = x2 + x2, u = u + u

The control inputs for fast and slow subsystems are designedindependently of each other.

. – p.17/46

Singular perturbation: important works

Singular Perturbation Methods in Control: Analysis and Design. P. V. Kokotovic,J. O’Reilly and H. K. Khalil, Academic press, 1986.

Adaptive neural network controller design for flexible joint robots using singularperturbation technique. S. S. Ge and I. Postlethwaite, Transaction of the instituteof measurement and control, 1995.

A neural network controller for flexible-link robots. A. Yesildirek, M. W. Vandergriftand F. L. Lewis, Journal of Intelligent and Robotic Systems, December 1996.

Singular Perturbations and Time Scales in Guidance and Control of AerospaceSystems: A Survey. D. Subbaram Naidu and Anthony J. Calise, Journal ofGuidance, Control, and Dynamics 2001.

Global exponential stability of competitive neural networks with different timescales. A. Mayer-Baese, S. S. Pilyugin and Y. chen. IEEE TNN May 2003.

. – p.18/46

Non-affine system

Non-affine systems have following forms

x = f(x, u) y = h(x, u)

x = f(x) + g(x, u)

. – p.19/46

Non-affine system - Important works

Control of nonlinear dynamical systems using neural networks:controlability and stabilization. A. U. Levin and K. S. Narendra,IEEE TNN, March 1993.

Control of nonlinear dynamical systems using neural networks -part II: Observability, identification and control, A. U. Levin and K.S. Narendra, IEEE TNN, January 1996.

Adaptive control using neural networks and approximate models,K. S. Narendra and S. Mukhopadhyay, IEEE TNN, May 1997.

Direct adaptive control of non-affine systems using multilayerneural networks. T. Zhang, S. S. Ge and C. C. Hang, Proc.American control conference, June 1998.

A new method for control of discrete nonlinear dynamic systemsusing Neural Networks, O. Adetona, E. Garcia and L. H. Keel,IEEE TNN, January 2000. . – p.20/46

Non-affine system: Important work

Adaptive control of non-affine nonlinear systems using neural-networks. S. S.Gee, T. H. Lee and J. Wang, IEEE symposium on intelligent control, July 2000.

A Neural-network based approximation method for discrete-time nonlinearservomechanism problem, Dan Wang and Jie Huang, IEEE TNN, May 2001.

Robust adaptive control of nonaffine nonlinear plants with small input signalchanges. O. Adetona, S. Sathananthan and L. H. Keel, IEEE TNN, March 2004.

Direct adaptive controller for nonaffine nonlinear systems using self-structuringneural networks. P. Jang-Hyun, H. Sung-Hoe, K. Seong-Hwan, S. Sam-Jun andP. Gwi-Tae, IEEE TNN, March 2005.

Observer-based direct adaptive fuzzy-neural control for nonaffine nonlinearsystems. Y. G. Leu, W. Y. Wang and T. T. Lee, IEEE TNN, July 2005.

. – p.21/46

Interconnected systems

An interconnected system of N-subsytems is represented as

xi = Aixi + biui + Di +N∑

j=1

fij(t, xj)

yi = hTi xi i = 1, 2, . . . , N

where for ith subsystem: xi ∈ Rni is the state vector,ui ∈ R1 is the control variable, yi ∈ R1 is the output,Di ∈ Rni is the disturbance vector, and fij(t, xj) ∈ Rni

contains the nonlinearities of ith subsystem and nonlinearinteractions with the other subsytems. The constantmatrices Ai, bi and hi are unknown constant matrices andnonlinear terms fij(t, xj) are assumed to satisfy‖fij(t, xj)‖ ≤ aij‖xj‖.

. – p.22/46

Interconnected systems

The control objective is to design local conrollers so thatindividual systems track the outputs of the correspondingreference models

xmi = Amixmi + bmiri; ymi = CTmixmi

Because of presence of interacting functions fij, the overall

system becomes nonlinear and difficult to control.

. – p.23/46

Inter-connected systems

Indirect adaptive control of interconected systems. K. A.Ossman, IEEE Trans. on Automatic control, 1989.

Decentralized adaptive control of interconnected system. P. A.Ioannu, IEEE Automatic control, April 1986.

Decentralized sliding mode adaptive controller design based onfuzzy neural networks for interconnected uncertain nonlinearsystems. Feipeng Da, IEEE TNN, November 2000.

Stability analysis of neural network interconnected systems, J. D.Hwang and F. H. Hsiao, IEEE TNN, January 2003.

Nonlinear adaptive control of interconnected systems usingneural networks. S. N. Huang, K. K. Tan and T. H. Lee, IEEETNN, january 2006.

. – p.24/46

Neural network models

Feedforward networksMultilayered networksRadial basis function networks

Recurrent NetworksHopfield networksMemory neuron networksDynamic Neural Networks

CMAC

Self-organizing map

. – p.25/46

Feedforward networks

Multilayer network:

One or more hidden-layers

Activation function - smoothfunctions - sigmoid

Ouput neurons may be linearor nonlinear

Most widely used in neurocontrollers.

y1

yn

u2

u1

um

φp

φ2

φ1

Radial Basis functions:

Single hidden layer

Activation form forms a cluster in the input space - gaussian orthin-plate spline

Linear output neurons

. – p.26/46

Feedforward networks - some works

Multilayer feedforward networks are universal approximators, K.Hornik, M. Stichcombe and H. White, Neural Networks, 1989.

Neural network control of robot manipulators, F. L. Lewis, IEEEInteligent systems, June 1996.

Neural network control of robot manipulators and nonlinearsystems, F. L. Lewis, S. Jagannathan and A. Yesildirek, Taylorand Francis, 1999.

Output feedback control of nonlinear systems using RBF neuralnetworks. S. Seshagiri and H. K. Khalil, IEEE TNN, January2000.

Adaptive output feedback control of uncertain MIMO systemsusing single hidden layer neural networks. N. Hovakimyan and A.J. Calise, American control conference, May 2002.

. – p.27/46

Recurrent Networks

Full-feedback connection

Learning algorithms: BPTT,RTRL

Computational complexity

y1(t)

y2(t)

ym(t)

y2(t + 1)

ym(t + 1)

z−1

z−1

z−1

z−1

y1(t + 1)

Locally recurrent globallyfeedforward

Reduced computational com-plexity

y1

u2

u1

. – p.28/46

RNN: important works

Neurocontrol of nonlinear dynamical systems with Kalman filtertrained recurrent networks. G. V. Puskorious and L. A.Feldkamp, IEEE TNN, March 1994

Memory Neuron Network for Identification and control ofdynamical systems. P. S. Sastry, G. Santharam and K. P.Unnikrishnan, IEEE TNN, march 1994.

Back propagation through adjoints for identification of nonlineardynamical systems using recurrent neural models. B. Srinivasan,U. R. Prasad and N. J. Rao, IEEE TNN, March 1994.

Diagonal recurrent neural network for dynamic system andcontrol. C. C. Ku and K. Y. Lee, IEEE TNN, January 1995.

Modelling, identification and stable adaptive control ofcontinuous-time nonlinear dynamical systems using neuralnetworks. M. Polycarpou and P. Ioannou, American controlconference, 1992. . – p.29/46

Dynamic neural network

The ouput of a dynamic neuron isdescribed by a differential equationas

xi = dixi +

n∑

j=1

aijφ(xj) + biu

where di < 0, φ(x) = tanh(x).

x1

xn

ubi

xiain

ai1

aii

R

xiP

di

single dynamic neuron

. – p.30/46

Dynamic Neural Networks

Dynamic neural controllers for induction motor. M. A. Brdys and G. J. Kulawski,IEEE TNN, March 1999.

Input-output linearization using dynamic recurrent neural network. A. Delgado, C.Kambhapati and K. Warwick, Mathematics and computer in simulation. Elsevier,August 1996.

Dynamic recurrent neural network for system identification and control, A.Delgado, C. Kambhampati, K. Warwick, IEE Proc. Control Theory andapplications, 1995.

Nonlinear adaptive trajectory tracking using dynamic neural network. A. S.Poznyak, W. Yu, E. N. Sanchez and J. P. Perez, IEEE TNN, November 1999.

Adaptive control of unknown plants using dynamic neural networks. G. A.Rovithakis and M. A. Christodoulo, IEEE SMC, March 1994.

. – p.31/46

CMAC networks

Its a perceptron like associativememory with overlapping receptivefields that is capable of learningmulti-dimensional nonlinearfunctions.

CMACs represent nonlinearfunctions y = f(x) using twoprimary mappings:

S : x ⇒ A

P : A ⇒ Y

where X is a continuouss−dimensional input space. A isthe association space and Y isone-dimensional output space.

P

x

y

Zero excitationNonzero excitation

space of all possibleinput vectors

x ⇒ AA ⇒ Y

SP

α W

Schematic of CMAC architecture

. – p.32/46

CMAC: important works

A New Approach to Manipulator Control: The Cerebellar Model ArticulationController (CMAC). J. S. Albus, Journal of dynamical systems, meausrement andcontrol, ASME, September 1975.

Discrete-time CMAC NN control of feedback linearizable nonlinear systems undera persistence of excitation. S. Jagannathan, IEEE TNN, january 1999.

CMAC Neural Networks for Control of Nonlinear Dynamical Systems: Structure,Stability and Passivity. S. Commuri and F. L. Lewis, Automatica, April 1997.

CMAC based adaptive critic self-learning control. C. S. Lin and H. Kim, IEEETNN, September 1991.

An adaptive control system design using memory based learning system. Y.Hirashima, Y. Iiguni and N. Adachi, International Journal of control, November1997.

Theory and development of higher-order CMAC neural networks. S. H. Lane, D.A. Handelman, and J. J. Gelfand, IEEE Control system magazine, April 1992.

. – p.33/46

Self-organizing map

SOM is characterized by the formation of atopographic map of input patterns in whichthe spatial locations of neurons in the latticeare indicative of statistical features containedin the input patterns.

Input

Array of neurons

Winner neuron

Competitive learning

A continuous input space ofactivation patterns is mapped onto adiscrete output space of neurons.

Gaussian activation functions

Input

. – p.34/46

SOM: some works

The self organizing map. T. Kohonen, Proc. IEEE March 1990.

Implementation of self-organizing neural networks forvisuo-motor control of an industrial robot. J. A. Walter and K. J.Schulten, IEEE TNN January 1993.

Local dynamic modeling with self-organizing maps andapplications to nonlinear system identification and control. J. C.Principe, W. Ludong and M. A. Motter, Proc. IEEE November1998.

A hybrid neural control scheme for visual-motor coordination. L.Behera, N. Kirubanandan, IEEE Control system magazine, 1999.

. – p.35/46

SOM: Some works

Multiple model based flight control design. G. K. Thampi, J. C.Prinicipe, M. A. motter, J. Cho and J. Lan, IEEE Symposium oncircuits and systems, 2002.

Modeling and inverse controller design for an unmanned airvehicle based on self organizing map. J. Cho, J. C. Principe, D.Erdogmus, M. A. Motter, IEEE TNN, March 2006.

. – p.36/46

Other neural control architectures

Model reference control

Internal model based control

Predictive control

Adaptive critic based control

. – p.37/46

Model reference control

Desired performance ofclosed loop system isspecified through a stablereference model M .

The control system attemptsto make the plant output yp(t)

match reference modeloutput yr(t) asymptotically.

Controller is the inverse plantmodel

yp

+

-

yr

r uPlant

C

M

er

M

. – p.38/46

Internal model based control

In this structure, a systemforward and inverse modelare used directly as elementswithin the feedback loop.

M - NN identifying plantmodel, C - NN controller(inverse plant model), F -linear filter for robustness

IMC has been widely used inprocess control.

+

ys

+e

-

uC

r ypPlant

Mym

-

F

In IMC, the difference between the system and model is used asa feedback signal.

. – p.39/46

Internal model based control

Neural network for nonlinear internal model control. K. J. Huntand D. Sbarbaro, IEE Proc. control theory and applications,September 1991.

Nonlinear control structures based on embedded neural systemmodels. G. Lightbody and G. Irwin, IEEE TNN, May 1997.

Nonlinear internal model control using neural networks:application to processes with delay and design issues. I. Rivalsand L. Personnaz, IEEE TNN, January 2000.

An approximate internal model-based neural control for unknownnonlinear discrete processes. H. Li and D. Hua, IEEE TNN, May2006.

. – p.40/46

Predictive control

A NN model M provides theprediction of the future plantresponse over specifiedhorizon.

The optimizer computes thecontrol u which minimizes fol-lowing cost function

ypu

yr

rC

ModelReference

Optimization M

Plant

ymu′

J =

N2∑

j=N1

(yr(t + j) − ym(t + j))2 +

N2∑

j=1

λj(u′(t + j − 1) − u′(t + j − 2))2

subject to the constraint of plant model. Another nn model C may be usedto mimic the action of optimizer, thereby obviating the need of a separateoptimizer routine once training is complete.

. – p.41/46

Adaptive critic based control

Optimatlity is introduced in nonlinear control.

Incorporates reinforcement learning and dynamic programming.

Adaptive critic utilizes an approximation of the optimal valuefunction to accomplish its control design - Approximate DynamicProgramming

An ADP structure consists of following main components:

Actor

Critic

Plant Model

Training loop for both actor and critic - AC algorithms

. – p.42/46

Adaptive critic based control

Actor Plant

Utility

CriticUpdate

ActorUpdate

u(t)x(t + 1) J(t + 1)

J(t)

x(t)

Critic 2

Critic 1

Schematic of critic based control design

. – p.43/46

Adaptive critic based control design

Both actor and critic are represented by some NN models.

Critic approximates the cost-to-go or value function J(t) andJ(t + 1) from the state information x(t) and x(t + 1) respectively.

Actor approximates control action.

The training is based on Bellman’s recursion formula given by

J(t) = U(t) + γJ(t+)

Actor weights are updated so as to minimize the present valuefunction J(t).

Critic weights are upated so that J(t) represents the actualcost-to-go function J(t).

. – p.44/46

ACD: important works

Adptive critic design. D. Prokhorov and D. C. Wunsch, IEEE TNNSeptember 1997.

Dynamic re-optimization of a fed-batch fermentor using adaptivecritic designs. M. S. Iyer and D. C. Wunsch, IEEE TNNNovember 2001.

Comparison of heuristic dynamic programming and dualheuristic programming adaptive critics for neurocontrol of aturbogenerator. G. K. Venayagamoorthy, R. G. Harley and D. C.Wunsch, IEEE TNN May 2002.

Implementation of adaptive critic-based neurocontrollers forturbogenerators in a multimachine power system. G. K.Venayagamoorthy, R. G. Harley and D. C. Wunsch, IEEE TNNSeptember 2003.

Gripper. S. Jagannathan and G. Salan, IEEE TNN March 2004. . – p.45/46

Summary

Following topic were covered in this lecture:

Direct and indirect adaptive control

Classification of nonlinear systems.

Neural network models

Neural control architectures

Literature survey

. – p.46/46