NON-LINEAR CONTROL OF OUTPUT PROBABILITY DENSITY FUNCTION FOR LINEAR ARMAX SYSTEMS

H. Yue, H. Wang

Control Systems Centre, University of Manchester Institute of Science and Technology, United Kingdom Institute of Automation, Chinese Academy of Sciences, Beijing 100080, China

Email: h.yue@umist.ac.uk; hong.wang@umist.ac.uk Fax: 44-161-2003858

Keywords: ARMAX model, stochastic system, probability density function (PDF), non-linear control.

Abstract

A new control algorithm has been developed for linear stochastic ARMAX systems subjected to arbitrary random inputs. The purpose is to make the shape of the output probability density function (PDF) to be as close as possible to a target PDF. A feedback control function is formulated to re-configure the open-loop linear system into a non-linear closed-loop system, where the vector of control parameters has been designed so as to minimize the difference between the output and the target PDFs. Issues of closed-loop stability are discussed.

1 Introduction

As required by a number of practical systems such as the particle size distribution control [1-3] and molecular weight distribution control [4-6] in polymerisation processes, 3D paper solids density distribution control in paper industries [7] and particle size distribution control in powder industries [8-9], etc., the shape control of the output probability density function (PDF) of general stochastic systems has been a subject of research in recent years [10-20]. For these systems, the main purpose of the controller design is to select a control input that can make the shape of the output PDF of a general stochastic system to follow a target probability density function. Compared with the traditional stochastic control methods such as minimum variance control and LQG control [21, 22], the PDF shaping methods have two main advantages: 1) the PDF contains full information of the system output including its mean and variance; 2) it is not required that the system is subjected to Gaussian noise only.

The development of output PDF shaping control are based on the following models:-

B-spline weights model [11-15]

Input-output ARMAX model [16-18]

Multi-layer neural-network model [19- 20]

So far most of the PDF shaping methods are based on the first group of modelling techniques, in which a B-spline neural network is used to approximate the measurable output PDF of

the considered stochastic system. Once the basis functions are chosen and fixed, the shape of the output PDF is determined by the B-spline expansion coefficients. In this context, PDF shaping control is transferred into the control of the B-spline coefficients vector. This leads to a number of algorithms that aim to minimize the following performance index

( )21 1( , ) ( )2 2

Tk kx kJ x u g x dx u Ruγ

∈Ω= − +∫ (1)

where is the control input, is the weighting matrix, ku)

R( , kx uγ is the output probability density function at sample

time when controlled by , k ku ( )g x is the given target probability density function, Ω is definition domain of x . The first term in equation (1) is an integral measure of the distance between the output PDF and its target. Although simple algorithms can be obtained from this group of research, some limitations of the B-spline methods shouldn’t be ignored: 1) the effectiveness of the closed-loop control relies on the selection of the basis functions; 2) the number of basis functions will increase dramatically if the system has multiple outputs; 3) the requirement of available (measurable) output PDFs may be too strict for some applications. Therefore, other models are also considered for PDF shaping control. For example, multi-layer neural network models have been used to approximate the PDFs when the system has more than one output [19, 20].

As for the input-output based approaches, normally a general ARMAX model is used, where the first step towards the controller design is to find a secondary model that links the PDF of the random input, the control input and some historical output measurements to the current conditional output PDF. This enables the direct optimisation being performed so as to find the required control input. Both linear [16] and non-linear ARMAX models [17, 18] have been considered for the PDF control or the minimum entropy control. In the linear case [16], it has been shown that when the control input u is related to the previous output of the system, only the position shift of the output PDF can be achieved, the shape of the output PDF cannot be changed at all. In the non-linear case [17], an approximated recursive relationship has been established for the convolution of the conditional output PDFs, which results in a numerical

k

Control 2004, University of Bath, UK, September 2004 ID-083

solution to the problem. However, the robustness of such a numerical approach needs to be further analysed.

The main idea of this paper is to find out how the conditional output PDF of a general linear ARMAX system can be controlled via a non-linear control function, which has an instant feedback from the system output. As the closed-loop system thus formulated is non-linear, it is expected that the conditional output PDF can be controlled to an arbitrary shape.

2 Output PDF model

Consider the following ARMAX model for linear SISO stochastic systems

1 1( ) ( )k kA q y B q u kω− −= + (2)

where

1

1

( ) 1n

ii

i

A q−

=

= −∑a q− (3)

1

0

( )m

jj

j

B q b q−

=

= ∑ − (4)

1ky R∈ is the system output, 1

ku R∈ is the control input, and

ia

jb

k

are known parameters of the system, and are

known structure orders, is the back shift unit operator.

n m1q−

ω is a random noise input that has a known probability density function ( )xωγ as defined on [ , ]x a b∈ with a and

being known and possibly infinite. In practice, b ( )xωγ can be estimated using kernel estimation theory [12].

Denote ( , )kx uγ as the conditional output probability density function and assume that its definition domain is also

[ , ]x a b∈ , then at sample time , the purpose of the control input design is to:-

k

find out a control input u that controls the shape of the output PDF of , namely

k

ky ( , )kx uγ , so that ( , )kx uγ is made as close as possible to a given

PDF ( )g x , which is also defined on [ , ]x a b∈ ;

ensure that the so formed control input is a feedback function of the current output, the historical outputs and the historical control inputs.

For this purpose, the following non-linear control function is proposed

( )1 10

1 , ,n m

k k k k k i k i j ki j

u y f y a y b ub

φ θ − −= =

= − − −

∑ ∑ j (5)

where ( )f ⋅ is a pre-specified non-linear function which is assumed to be invertible with respect to ; ky kφ groups all the past inputs and outputs as

( )1 1, , , , ,k k k n k k my y u uφ − − − −= (6)

This means kφ is known at time . k kθ is a parameter vector to be designed, the size of which depends on the controller formulation. For example, if the following non-linear structure is adopted for ( )f ⋅

31( , , ) sin( )k k k k kf y y yφ θ α β −= − (7)

then a pair of parameters ( ,α β ) are to be designed.

Taking equation (6) into equation (5), is further represented as

ku

( )( )0

1 , ,k k k k ku y f yb kφ θ φ= − − (8)

The controller in equation (8) has non-linear nature because of ( )f ⋅ . Therefore, the closed-loop system composed by equations (2)-(6) is also non-linear. This allows the shape control of the conditional output probability density function to go beyond the shift of ( )xωγ only. Indeed, by substituting equation (8) into the open-loop equations (2-4), it can be derived that the closed-loop system should have the following format

( ), ,k k k kf y φ θ ω= (9)

Since it has been assumed that function ( )f ⋅ is invertible with respect to , then can be solved from equation (9) to give

ky ky

1( , , )k k ky f kω φ θ−= (10)

which reveals a general non-linear relationship between and

ky

( ), ,k k kω φ θ . As the probability density function of kω is assumed known to be ( )xωγ , the conditional probability density function of can be readily formulated from equation (10) to read [23]

ky

( )( ) ( ), ,( , ) , , , [ , ]k k

k k k

f xx u f x x a

xω

φ θγ γ φ θ

∂= ∈

∂b (11)

From equation (11) it is concluded that selecting a control input to shape ku ( , )kx uγ is equivalent to choosing a proper parameter vector kθ so that ( , )kx uγ is controlled to its target once the control function ( )f ⋅ is selected. This indicates that one can simply denote ( , )kx uγ as ( , )kxγ θ and describe the output PDF as

( )( ) ( ), ,( , ) , , , [ , ]k k

k k k

f xx f x x a b

xω

φ θγ θ γ φ θ

∂= ∈

∂ (12)

Equation (12) is the secondary model of the output PDF showing that ( , )kxγ θ is decided by ( )f ⋅ and kθ .

Control 2004, University of Bath, UK, September 2004 ID-083

3 Closed-loop PDF control system

3.1 Controller design

The purpose of controller design is now transformed into the formulation of the non-linear function ( )f ⋅ and the selection of the parameter vector kθ , at sample time k , so that the following performance function is minimized.

( )2

21 ( , ) ( )2 2

b

k ka

1J x g x dx Rγ θ= − +∫ u (13)

Such an optimisation is subjected to equation (12) and can be reformulated to be

( )( ) ( )2

, ,1 , , ( )2

b k kk ka

f xJ f x g x dx

xω

φ θγ φ θ ∂

= ∂∫

−

( )(2

0

1 1 , ,2 k k k k kR y f y

bφ θ φ

+ − −

) (14)

The minimum solution to equation (14) may be obtained by

0k

Jθ∂

=∂

(15)

in certain cases. However, due to the complexity of the integral nature involved in equation (14), a compact solution to equation (15) may be difficult to obtain. Instead, a gradient search can be applied, i.e.

1

1l

l lJ

θ θ

θ θ λθ

−

−=

∂= −

∂ (16)

where ( l ) is the number of iteration steps and l k≥ λ is the gradient factor. Although equation (16) only provides a local minimum solution to the performance index in equation (14), there seems to be no better choices when ( , )k kf y , kφ θ only takes a general form of a non-linear function. In fact, if the structure of ( , , )k k kf y φ θ can be parametized in the design, some state-of-the-art optimal solution by equation (15) may be available based on the specific formulation of

( ,f y , )kk kφ θ .

In some situations, the target probability density function ( )g x may be difficult to establish from real systems, then the

control algorithm design should be performed by minimizing the entropy of the system output or the entropy of the output tracking errors. The minimum entropy control of the system output can be realized by defining the following performance function

( ) 21 ( , ) log ( , )2 2

b

k ka

1kJ x x dxγ θ γ θ= ∫ Ru+ (17)

where the first term is the entropy of as defined in probability theory. As the entropy index is used to describe the uncertainty of general non-Gaussian systems, the

minimum solution to equation (17) is believed to reduce the uncertainty of the closed-loop system [15,18].

ky

3.2 Stability issues

As described in section 2, the noise probability density function ( )xωγ is defined on [ , ]x a b∈ with and being known or possibly infinite. In the following, these two cases will be discussed separately for the analysis of the closed-loop stability.

a b

A. Noise with bounded PDF

When both and are bounded, it means that the noise term

a bkω is defined on a bounded domain. From the closed-

loop formulation in equation (9), it also implies that the pre-selected non-linear function ( , , )k k kf y φ θ

( , ,k kf y

must be bounded. In this case, the closed-loop stability can be realized by making sure that the inverse of )kφ θ (with respect to

!) is also uniformly bounded for the closed-loop equation (9). In fact, under this circumstance, we have

ky

1( , , ) ,k k k ky f kω φ θ− θ= < +∞ ∀ (18)

Equation (18) indicates that is uniformly bounded. It can be seen from equations (2) – (4) and (9) that

ky

1 1( ) ( ) ( , ,k k k kB q u A q y f y )kφ θ− −= − (19)

Since both and ky ( , , )k k kf y φ θ

ku are uniformly bounded,

when is stable, is also uniformly bounded. The conditions for the closed-loop stability can be summarized in the following theorem.

1(B q− )

Theorem 1. Suppose that i) the polynomial is stable, and that 1(B q− )ii) and b are bounded, athen the closed-loop system described by equations (8) and (9) are uniformly bounded if ( , , )k k kf y φ θ is selected such that the following inequality

1( , , ) ,k k k kf ω φ θ− < +∞ ∀θ (20)

is satisfied.

B. Noise with unbounded PDF

If at least one of or is infinite, the condition of critical closed-loop stability does not exist because the main task is to make the conditional output PDF to follow a desired

a b

( )g x that is defined on an infinite interval. This would be similar to the case in linear minimum variance control [21] when ( )xωγ is a Gaussian function, where the uniform boundness of the output cannot be guaranteed as is a Gaussian random process defined on an infinite domain

ky

Control 2004, University of Bath, UK, September 2004 ID-083

( ),−∞ +∞

ky. In this case, only the boundness of the mean value

of will be discussed.

E

When either a or is infinite, although itself is within an unbounded interval, its mean value can still be made uniformly bounded under certain conditions. Assume that the mean value of the noise term is bounded, i.e.

b ky

E kω < +∞ (21)

then from equation (9) it can be seen that

( )E , , k k kf y φ θ < +∞ (22)

If we can assume that ( , , )k k kf y φ θ

ky is selected such that it is

monotonic with respect to , then from equations (10) and (22), the monotonic inverse of ( , , )k k kf y φ θ also has the uniform boundness as

( )1E E , , k k k ky f ω φ θ−= < +∞

)

(23)

As a result, if is stable, then from equations (19), (22) and (23), it can be concluded that

1(B q−

( )11

1 E ( ) ( , , )( )k k k ku A q y f y

B qφ θ−

−

= −

(24) k < +∞

)

This result can be summarized as follows.

Theorem 2. Suppose that i) the polynomial is stable, 1(B q−

ii) E kω < +∞ , and iii) ( , , )k k kf y φ θ is monotonic with respect to , kythen regardless of kθ , the mean values of the input and output of the closed-loop system described by equations (8) and (9) are uniformly bounded.

4 An example

Consider the following linear system

1 1 0k k ky a y b u kω−= + + (25)

where it is assumed that the probability density function of kω is given by

( ) , [0, )xx e xωγ−= ∈ +∞ (26)

which corresponds to and . The target probability density function is given to be

0a = b = +∞

2

( ) 2 xg x xe−= (27)

In this case, one can construct the following control function

( )1 10

1 ( , , )k k k k k ku y f y a yb

φ θ −= − − (28)

where

( )2 21( , , ) 1k k k k k kf y y yφ θ −= + θ (29)

and 1k Rθ ∈ .

Using equation (12), the conditional output probability density function is developed to be

( ) ( )2 2112

1( , ) 2 1 k kx yk k kx y xe

θγ θ θ −− +

−= + (30)

For this simple example, we can just select

21

11k

kyθ

−

=+

(31)

then the non-linear function reduces to 2( , , )k k k kf y yφ θ = (32)

and the control input and the output of the closed-loop system turn out to be

( 21 1

0

1k k k ku y y a y

b −= − − )

k

(33)

2ky ω= (34)

The output probability density function is controlled to be 2

( , ) 2 xkx xeγ θ −= (35)

Therefore, the target distribution in equation (27) is achieved via the non-linear control law in equations (28), (29) and the tuning rule in equation (31). The first term in the performance index (13) is zero and the second term forms J, which is only related to the input noise, i.e.,

( )212

0

(1 )2 k kRJ ab

ω ω= − − (36)

As J in equation (36) might not be the optimal solution for the whole performance index in equation (13), the gradient approach in equation (16) can be used, in which the gradient is represented as

( ) 21

1

22 21 1 0

4 1 l

l

xl l

J y x e α

θ θ

αθ

−

−

+∞ −− −

=

∂= +

∂ ∫ dx

dx

dx

dx

( ) 21(1 )2 2

1 04 1 l x

ly x e α −+∞ − +

−− + ∫ ( ) 2

122 2 41 1 0

4 1 l xl ly x e αα −

+∞ −− −− + ∫

( ) 21(1 )2 4

1 1 04 1 l x

l ly x e αα −+∞ − +

− −+ + ∫(2 2

1 1 120

l l l l l lR y y y a yb

α α− −− − − )1− (37)

in which

( )221 11l ly 1lα θ− −= + − (38)

Control 2004, University of Bath, UK, September 2004 ID-083

5 Conclusions

In this paper, a new parametric control algorithm has been established that is capable of controlling the shape of the conditional output probability density function for a linear stochastic system represented by an ARMAX model. The system is subjected to any arbitrary random input with a known probability density function. The obtained controller is generally non-linear and its embedded parameter vector can be selected so as to control the conditional PDF of the output to be close to a given target PDF. This result is the extension of the recent work published by Wang and Zhang [16] where only the shift control of the probability density function of the conditional output for linear systems is achieved.

Acknowledgements

The authors would like to thank the financial support of the Chinese NSFC grant 60128303 and UK's Leverhulme Trust grant F/00038/D.

References

[1] T. J. Crowley, E. S. Meadows, E. Kostoulas, F. J. Doyle III. “Control of particle size distribution described by a population balance model of semibatch emulsion polymerisation”, Journal of Process Control, 10, pp. 419-432, (2000).

[2] J. Flores-Cerrillo, J. F. MacGregor, “Control of particle size distributions in emulsion semibatch polymerization using mid-course correction polices”, Ind. Eng. Chem. Res., 41, pp. 1805 –1814, (2002).

[3] C. D. Immanuel, F. J. Doyle III. “Open-loop control of particle size distribution on semi-batch emulsion copolymerisation using a genetic algorithm”, Chem. Eng. Sci., 57, pp. 4415 – 4427, (2002).

[4] T. L. Clarke-Pringle, J. F. MacGregor. “Optimization of molecular-weight distribution using batch-to-batch adjustments”, Ind. Eng. Chem. Res., 37, pp. 3660–3669, (1998).

[5] A. Echevarria, J. R. Leiza, J. C. de la Cal, J. M. Asua. “Molecular weight distribution control in emulsion polymerisation”, AIChE Journal, 44, pp.1667–1679, (1998).

[6] M. Vicente, S. BenAmor, L. M. Gugliotta, J. R. Leiza, J. M. Asua. “Control of molecular weight distribution in emulsion polymerization using on-line reaction calorimetry”, Ind. Eng. Chem. Res., 40, pp. 218–227, (2001).

[7] G. A. Smook. Handbook for Pulp and Paper Technologists, Angus Wilde Publications, (1998).

[8] R. A. Eek, O. H. Bosgra. “Controllability of particulate processes in relation to the sensor characteristics”, Powder Technology, 108, pp.137-146, (2000).

[9] H. J. C. Gommeren, D. A. Heitzmann, J. A. C. Moolenaar, B. Scarlett. “Modelling and control of a jet

mill plant”, Powder Technology, 108, pp.147-154, (2000).

[10] M. Karny. “Towards fully probabilistic control design”, Automatica, 12, pp. 17119-1722, (1996).

[11] H. Wang. “Robust control of the output probability density functions for multivariable stochastic systems”, IEEE Transaction on Automatic Control, 44, pp. 2103-2107, (1999).

[12] H. Wang. Bounded Dynamic Stochastic Systems: Modelling and Control, Springer-Verlag London Limited, (2000).

[13] H. Wang, H. Baki and P. Kabore. “Control of bounded dynamic stochastic distributions using square root models: an applicability study in papermaking system”, Trans. Inst. of Measurement and Control, 23, pp. 51-68, (2001).

[14] H. Wang, H. Yue. “A rational spline model approximation and control of output probability density function for dynamic stochastic systems”, Trans. Inst. of Measurement and Control, 2003, 25, pp. 93-105, (2003).

[15] H. Wang. “Minimum entropy control for non-Gaussian dynamic stochastic systems”, IEEE Trans. on Automatic Control, 47, pp.398 – 403, (2002).

[16] H. Wang, J. H. Zhang. “ Bounded stochastic distribution control for pseudo ARMAX systems”, IEEE Trans. on Automatic Control, 46, pp. 486-490, (2001).

[17] H. Wang. “Control of conditional output probability density functions for general non-linear and non-Gaussian dynamic stochastic systems”, IEE Proc. Control Theory and Applications, 150, pp. 55 – 60, 2003.

[18] H. Yue and H. Wang. “Minimum entropy control of closed-loop tracking errors for dynamic stochastic systems”, IEEE Trans. on Automatic Control, 48, pp. 118 – 122, (2003).

[19] H. Wang. “Multivariable output probability density function control for non-Gaussian stochastic systems using simple MLP neural networks”, Proc. IFAC Int. Conf. on Intelligent Control Systems and Signal Processing, University of Algarve, Portugal, April 8-11, 2003, pp.84-89, (2003).

[20] W. Wang, H. Wang, H. Yue. “MIMO probability density function control using simple MLP neural networks”, The 4th IASTED Int. Conf. on Modelling, Simulation, and Optimization, Kauai, Hawaii, August 17-19, 2004. (in print)

[21] K. J. Astrom. Introduction to Stochastic Control Theory, Academic press, New York, (1970).

[22] K. J. Astrom and B. Wittenmark. Adaptive Control, Addison-Wesley, Reading, Mass, (1988).

[23] J. L. Melsa and P. S. Andrew, An Introduction to Probability and Stochastic Processes, T. Kailath (ed.), Prentice Hall, Englewood Cliffs, (1973).

Control 2004, University of Bath, UK, September 2004 ID-083