ARTICLE IN PRESS

0967-0661/$ - se

doi:10.1016/j.co

$A shorter

predictive cont

presented at th

of the finalist fo�CorrespondE-mail addr

Control Engineering Practice 15 (2007) 839–850

www.elsevier.com/locate/conengprac

Evaluation study of an efficient output feedback nonlinearmodel predictive control for temperature tracking in

an industrial batch reactor$

Zoltan K. Nagya,�, Bernd Mahnb, Rudiger Frankec, Frank Allgowerd

aLoughborough University, Chemical Engineering Department, Loughborough LE11 3TU, UKbBASF Aktiengesellschaft, Ludwigshafen, GermanycABB Corporate Research, Ladenburg, Germany

dUniversity of Stuttgart, Stuttgart, Germany

Received 7 October 2005; accepted 16 May 2006

Available online 26 July 2006

Abstract

The paper illustrates the benefits of nonlinear model predictive control (NMPC) for the setpoint tracking control of an industrial batch

polymerization reactor. Real-time feasibility of the on-line optimization problem from the NMPC is achieved using an efficient multiple

shooting algorithm. A real-time formulation of the NMPC that takes computational delay into account is described. The control relevant

model for the NMPC is derived from the complex-first principles model and is fitted to the experimental data using maximum likelihood

estimation. A parameter adaptive extended Kalman filter (PAEKF) is used for state estimation and on-line model adaptation. The

performance of the NMPC implementation is assessed via simulation and experimental results.

r 2006 Elsevier Ltd. All rights reserved.

Keywords: Industrial control; Maximum likelihood estimation; Multiple shooting algorithm; Nonlinear model predictive control; Parameter adaptive

extended kalman filter; Parameter estimation; State estimation; Real-time control

1. Introduction

The inherent advantages of batch processes, includingtheir ability to produce multiple related products in the samefacility, as well as their ability to handle variations in feedstocks, product specifications and market demand patterns,makes them well suited for the manufacture of low-volume,high-value products (Bonvin, 1998; Bonvin, Srinivasan &Hunkeler, 2006; Nagy & Braatz, 2003a, b). For thesereasons, batch processes are the production scheme of choicefor the pharmaceutical, biotechnology, specialty chemical,consumer products, agricultural chemical, and microelec-tronics industries. The production of these high value-added

e front matter r 2006 Elsevier Ltd. All rights reserved.

nengprac.2006.05.004

version of the paper with the title ‘‘Nonlinear model

rol of batch processes: an industrial case study’’ was

e IFAC World Congress at Prague in 2005, and was one

r the ‘‘Best application paper award’’.

ing author. Tel.: +441509 222516; fax: +44 1509 223923.

ess: Z.K.Nagy@lboro.ac.uk (Z.K. Nagy).

chemicals, today contributes a significant and growingportion of the revenue and earnings of the chemical processindustries. Due to the highly competitive and profit-drivennature of today’s process industries, Nonlinear modelpredictive control (NMPC) techniques are becoming increas-ingly accepted, being one of the approaches that inherentlycan cope with process constraints, nonlinearities, anddifferent objectives derived from economical or environ-mental considerations (Bequette, 1991; Henson, 1998;Morari & Lee, 1997). However, despite the significant andcontinuously increasing importance of batch processes, thenumber of NMPC applications is significantly lower than inthe case of continuous processes (Qin & Badgwell, 2003).Although the inherent nonlinearity of batch processessuggests NMPC as a natural choice for the advanced controlof these systems, most industrial NMPC vendors for exampledo not support typical batch NMPC problems. A fewexceptions include the most well-known Cybernetica’sCENIT software (Cybernetica), which has been successfully

ARTICLE IN PRESSZ.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850840

used for several industrial batch and semi-batch processes,including polymerisation (see Cybernetica website). Thesignificantly lower number of NMPC applications, especiallyfor batch systems, can be explained mainly by the specialfeatures of batch processes that make their control verychallenging (Bonvin et al., 2006). Generally, in batch processoperation, two different control problems arise:

(a)

End-point property control (setpoint optimization ap-proach). In the case of batch processes the realeconomic objective is usually related to the productquality at the end of the batch, leading to theformulation of a control problem in terms of economicor performance objective at the end of the batch, whichis always implemented in a shrinking horizon ap-proach.

(b)

Setpoint tracking, when off-line or on-line determined,usually time-varying setpoint trajectories have to befollowed in a moving or shrinking (or combination ofthe two) horizon approach. In this case usually aquadratic (least-squares type) objective function isused.

In most of the applications in advanced batch control thefirst problem is solved off-line through open-loop optimalcontrol, and the objective of the on-line control is to dealwith the tight temperature regulation. Although this is atypical control problem, it presents a major differencecompared to the operation of continuous processes, sinceimportant properties such as Lyapunov stability are nolonger defined. Controller performance can only beassessed based on the chosen objective function, existingconstraints and robustness against model/plant mismatchor control implementation uncertainties. Additionally, inthe case of continuous processes computational burden canbe reduced by choosing a shorter control horizon thenprediction horizon, whereas in the case of batch NMPC thecontrol and prediction horizons should be chosen equal, toavoid large deviations of the predicted quantities from their(usually time-varying) setpoints due to the transientcharacter of the process, increasing significantly thecomputational demand.

To decrease computational complexity another categoryof control approaches have been proposed (Srinivasan,Bovin, Visser, & Palanki, 2003; Welz, Srinivasan, & Bovin,2006), which is based on simple representation of theprocess/controller model, and the application of Pontrya-gin’s Maximum Principle to derive the necessary conditionsof optimality (NCO), which is then tracked using currentmeasurements. Several successful application of the ap-proach to batch processes have been reported (e.g. Bodizset al., 2006; Bonvin et al., 2006). Although this approachsignificantly reduces the on-line computational burden, andprovides an alluring robust control framework, it requiressignificant off-line effort to derive the solution model andthe NCO. Additionally, current trends in the chemicalindustries show that companies tend to derive detailed first-

principle models for most of their important process, forsimulation and operator-training purposes. NMPC offers ageneric framework to incorporate existing models in auniform and flexible framework.The paper presents an experimental and simulation

evaluation study of NMPC for the temperature control ofa pilot scale reactor. The main objective of the research wasto provide a comprehensive evaluation of the implementa-tion complexity of NMPC in a real industrial framework andto convey to the readers the massage that in order to benefitfrom the advantages of NMPC a judicious consideration ofmultiple implementation problems, such as modelling,identification, computation, software development, estima-tion tuning and consideration of uncertainties are allrequired. The paper describes an efficient real time NMPCtool, which is applied to a pilot-scale industrial batchpolymerisation reactor. The tool exploits the advantages ofan efficient optimization algorithm based on multipleshooting technique (Diehl, 2001; Diehl et al., 2002; Franke,Arnold, & Linke) to achieve real-time feasibility of the on-line optimization problem involved in the NMPC, even inthe case of the large control and prediction horizon. Thecomplex first principle model of the process is used off-line todetermine the optimal temperature profile that provides therequired quality and conversion in minimum time, and hasalso been used for on-line setpoint adaptation for end-pointquality control (Nagy, Allgower, Franke, Frick, & Mahn,2004), however this paper focuses on the setpoint trackingperformance of the proposed NMPC tool. More details onthe off-line optimization of a similar batch polymerizationprocess using the dynamic simulation environment gPROMScan be found elsewhere (PSE; Van Overschee & Van Brempt,2000). In this paper the NMPC is used for tight setpointtracking of the temperature profile. Based on the availablemeasurements the complex model is not observable hencecannot be used directly in the NMPC strategy. To overcomethe problem of unobservable states, a grey-box modellingapproach is used, similarly to other industrial NMPCsolutions (e.g. Cybernetica, IPCOS, Aspen Target, etc.),where some unobservable parts of the model are describedthrough nonlinear empirical relations, developed from thedetailed first-principles model. The resulted control-relevantmodel is fine tuned using experimental data and maximumlikelihood estimation. A parameter adaptive extended Kal-man filter (PAEKF) is used for state estimation and on-lineparameter adaptation to account for mode/plant mismatch.

2. Computationally efficient real-time output feedback

NMPC

2.1. Problem formulation

Nonlinear model predictive control is an optimization-based multivariable constrained control technique that usesa nonlinear dynamic model for the prediction of theprocess outputs. At each sampling time the model isupdated on the basis of new measurements and state

ARTICLE IN PRESSZ.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850 841

variable estimates. Then the open-loop optimal manipu-lated variable moves are calculated over a finite predictionhorizon with respect to some cost function, and themanipulated variables for the subsequent predictionhorizon are implemented. Then the prediction horizon isshifted or shrunk by usually one sampling time into thefuture, and the previous steps are repeated.

The optimal control problem to be solved on-line inevery sampling time in the NMPC algorithm can beformulated as:

Problem P1:

minuðtÞ 2 U

HðxðtÞ; uðtÞ; yÞ, (1)

subject to:

_xðtÞ ¼ f ðxðtÞ; uðtÞ; yÞ, (2)

yðtÞ ¼ gðxðtÞ; uðtÞ; yÞ, (3)

xðtkÞ ¼ xðtkÞ; xðt0Þ ¼ x0, (4)

hðxðtÞ; uðtÞ; yÞp0; t 2 ½tk; tF �, (5)

whereH is the performance objective, t is the time, tk is thetime at sampling instance k, tF is the final time at the end ofprediction, xðtÞ 2 Rnx is the nx vector of states, u(t)AU isthe nu set of input vectors, yðtÞ 2 Rny is the ny vector ofmeasured variables used to compute the estimated statesxðtkÞ, y 2 Y � Rny is the ny vector of possible uncertainparameters, where the set Y can be either defined by hardbounds or probabilistic, characterized by a multivariateprobability density function. The function f : Rnx �U�Y! Rnx is the twice continuously differentiable vectorfunction of the dynamic equations of the system, g :Rnx �U�Y! Rny is the measurement equations func-tion, and h : Rnx �U�Y! Rc is the vector of functionsthat describe all linear and nonlinear, time-varying or end-time algebraic constraints for the system, where c denotesthe number of these constraints. The objective function canhave the following general form:

HðxðtÞ; uðtÞ; yÞ ¼MðxðtF Þ; yÞ þZ tF

tk

LðxðtÞ; uðtÞ; yÞdt. (6)

We assume that H : Rnx �U�Y! R is twice con-tinuously differentiable, thus fast optimization algorithms,based on first and second order derivatives may beexploited in the solution of (6). The form of (6) is generalenough to express a wide range of objectives encounteredin NMPC applications (moving or shrinking horizonapproach on regulation and/or setpoint tracking, directminimization of the operation time, optimal initial condi-tions, multiple simultaneous objectives, treatment of softconstraints, terminal penalty term for stability, etc.). Forbatch processes with end-point optimization the objectiveusually reduces to the Mayer form (Lð�Þ ¼ 0), however theLagrange term (Lð�Þ) still may be used, e.g. to implementsoft constraints on control rate. In NMPC the optimizationproblem (1)–(5) is solved iteratively on-line, in a moving

(receding) horizon (tFotf) or shrinking horizon (tF ¼ tf)approach, where tf is the batch time.

2.2. Efficient solution of the NMPC optimization via direct

multiple shooting

Considering the discrete nature of the on-line controlproblem, the continuous time optimization probleminvolved in the NMPC formulation is solved by formulat-ing a discrete approximation to it, that can be handled byconventional nonlinear programming solvers (Biegler,2000). The time horizon t 2 ½t0; tf � is divided into N

equally spaced time intervals Dt (stages), with discrete timesteps tk ¼ t0+kDt, and k ¼ 0,1,y,N. Model equations arediscretized,

xkþ1 ¼ f kðxk; uk; yÞ (7)

and added to the optimization problem as equalityconstraints. Problem P2 gives the general discrete timeformulation of Problem P1.Problem P2:

minuk ;ukþ1;...;ukþNp

fMkþNp ðxkþNp; yÞ þ

XkþNp

j¼k

Ljðxj ; uj; yÞg (8)

subject to:

Gkðxk; uk; yÞ ¼ 0, (9)

Hkðxk; uk; yÞp0, (10)

where Np is the number of stages in the prediction horizon[tk, tF], Gk : R

nx �U�Y! Rnxþny corresponds to allequality constraints resulted from (7) and (3), Hk : R

nx �

U�Y! Rcþ2nu is the vector function of all inequalityconstraints (5), including the constraints on the inputsconsidering. We consider here that the set of possibleinputs is given as hard bounds. It is assumed that the vectorfunctions G and H are twice continuously differentiable.A very efficient solution technique for problem (8)–(10)

is based on the multiple shooting approach (Diehl, 2001;Diehl et al., 2002; Franke et al.). The direct multipleshooting procedure consists of dividing up the time interval½tk; tF � into M subintervals ½ti; tiþ1� via a series of gridpoints tk ¼ t0ot1ot2o � � �otM ¼ tF . Note that the gridpoints do not necessary correspond to the discretizationpoints (N) in the definition of problem P2. Using a localcontrol parameterizations a shooting method is performedbetween successive grid points (see Fig. 1). We assume thatthe control ukð�Þ is parameterized as a piecewise constantvector function, ukðtÞ ¼ ukji, or simply uðtÞ ¼ ui fort 2 ½ti; tiþ1�, however any parameterization could be usedwith similar problem formulation. The solution of theODE on the M intervals are decoupled by introducing theinitial values oi of the states at the multiple shooting nodesti as additional optimization variables. The differentialequations and cost on these intervals are integrated inde-pendently during each optimization iteration, based on thecurrent guess of the control, and initial conditions oi. The

ARTICLE IN PRESS

u0

t0

x0

tft1

Initial profile

Final profile

Fig. 1. Illustration of the multiple shooting approach.

Z.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850842

continuity/consistency of the final state trajectory at theend of the optimization is enforced by adding consistencyconstraints to the nonlinear programming problem. Theadditional interior boundary conditions, are incorporatedinto one large nonlinear programming problem (NLP) tobe solved, which is given in a simplified form below:

Problem P3:

minv

Hðv; yÞ subjecttoGðv; yÞ ¼ 0

Hðv; yÞ � 0

((11)

where

Hðv; yÞ ¼MðoMþ1; yÞ þXMi¼0

Liðoi; ui; yÞ, (12)

and the optimization variable v contains all multipleshooting state variables and controls

v ¼ ½o0; u0;o1; u1; . . . ;oM�1; uM�1;oM �. (13)

The discretized initial value problem and continuityconstraints are included in the equality constraints:

Gðv; yÞ ¼

o0 � xðtkÞ;

oiþ1 � xiðtiþ1;oi; uiÞ

y� gðoi; ui; yÞ

264

375 ¼ 0, (14)

and the inequality constraints are given by Hðv; yÞ ¼Hðoi; ui; yÞp0 for i ¼ 0; 1; . . . ;M.

2.3. Real-time NMPC algorithm

The solution of problem P3 requires a certain, usuallynot negligible, amount of computation time dk, while thesystem will evolve to a different state. In this case theoptimal feedback control uðtkÞ ¼ ½u0jtk

; u1jtk; . . . ; uNpjtk

�

computed in moment tk corresponding to the informationavailable up to this moment, will no longer be optimal.Computational delay dk has to be taken into considerationin real-time applications. In the approach used here, inmoment tk, first the control input from the second stage ofthe previous optimization problem u1jtk�1

(which corre-sponds to the first stage of the current optimization) isinjected into the process, and then the solution of thecurrent optimization problem is started, with fixed u0jtk

¼

u1jtk�1After completion, the optimization idles for the

remaining period of t 2 ðtk þ dk; tkþ1Þ, and then at thebeginning of the next stage, at moment tkþ1 ¼ tk þ Dt, u1jtk

is introduced into the process, and the algorithm isrepeated. This approach requires real-time feasibility forthe solution of each open-loop optimization problems(dkpDt).

2.4. Solution tool for efficient NMPC implementation

The solution of Problem P2 is performed using anNMPC tool (Nagy et al., 2004) based on the sequentialquadratic programming (SQP) type optimizer HQP(Franke et al.), which is used in conjunction with theimplicit differential-algebraic equation (DAE) solver,DASPK, for robust and fast solution of the modelequations (Li & Petzold, 1999). The NMPC tool developedincludes a number of desirable features. In particular, theNMPC is based on first-principle or grey-box models, andthe problem formulation is performed within Matlabs

(The Mathworks Inc.), which is the most widely usedmodeling and control environment for control engineers.The model used in the controller has to be developed in theform of Simulinks (The Mathworks Inc.) ‘‘mex S-function’’ using C language. The S-function interface ofthe optimization tool provides convenient and fastconnectivity with Matlabs. The NMPC implementationoffers very efficient approach, based on a multiple shootingalgorithm, that exploits the special structure of optimiza-tion problem that arises in NMPC. The software uses lowrank updates of the approximation of the LagrangianHessian of the nonlinear subproblems combined with asparse interior point algorithm for the efficient treatment ofthe linear-quadratic subproblems in the nonlinear SQPiterations. Bounds and inequality constraints are handledusing a barrier method and line search is used for globalconvergence of the SQP iterations. The software packageimplements the real-time approach, and also provides anOPC connectivity, making it appropriate for rapid real-time prototyping of NMPC algorithms in an industrialenvironment.

ARTICLE IN PRESS

Fig. 2. Schematic representation of the batch reactor with the heating/

cooling system.

Z.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850 843

2.5. State estimation using parameter adaptive extended

kalman filter

Proper state estimation is crucial for the success of theNMPC application. Extended Kalman filter (EKF) hasbeen widely used in process control applications, howeverits performance strongly depends on the accuracy of themodel. To avoid highly biased model predictions, some ofthe model parameters are estimated together with thestates, leading to a parameter adaptive EKF formulation(PAEKF) (Valappil & Georgakis, 2002). Define y0 � y asthe subset of the estimated parameters from the parametervector, and y009y\y0 the set of the remaining parameters.The augmented state vector in this case is given byX ¼ ½x; y0�T, and the augmented model used for estima-tion is given by

_X ¼ ½f ðX; u; y00Þ; 0�T þ ½w; wy0 �T, (15)

where w, and wy0 are zero-mean Gaussian white noisevariables. The time-varying state space matrix of the locallylinearized augmented model is defined as follows:

AðtkÞ ¼

@f ðXðtkÞ;uðtkÞ;y00Þ

@x@f ðXðtkÞ;uðtkÞ;y

00Þ

@y0

0 0

" #. (16)

The time update of the states and state covariance arepropagated by integrating the model equations togetherwith the covariance propagation equation for one samplingtime ðt 2 ½tk�1; tk�Þ:

_X ¼ ½f ðX; u; y00Þ; 0�T, (17)

_PðtÞ ¼AðtÞPðtÞ þ PðtÞATðtÞ þQðtÞ, (18)

with t 2 ½tk�1; tk� and initial conditions Xðtk�1Þ and Pðtk�1Þ

obtained from the last estimation, and the jacobian Agiven by (16). Define the solutions of (17)–(18) as X

�ðtkÞ

and P�ðtkÞ, respectively. With these values the Kalmangain K is computed, and then the measurement updatestage is performed according to:

KðtkÞ ¼ P�ðtkÞ ~CTðtkÞð ~CðtkÞP

�ðtkÞ ~CTðtkÞ þ RÞ�1, (19)

PðtkÞ ¼ ðI� KðtkÞ ~CðtkÞÞP�ðtkÞ, (20)

FK ðtkÞ ¼ KðtkÞðymðtkÞ � gðX�ðtkÞ; uðtkÞ; y

00ÞÞ, (21)

XðtkÞ ¼ X�ðtkÞ þ F K ðtkÞ, (22)

where ym(tk) corresponds to the measurements obtainedfrom the real process at time tk, FK(tk) is the Kalman filtercorrection factor, and ~C is the jacobian of the measurementequations with respect to the augmented states:

~CðtkÞ ¼@gðXðtÞ; uðtÞ; y

00Þ

@X

!XðtkÞ;uðtkÞ

. (23)

The estimated states in (22) are used as the initial valuefor the model prediction stage in the optimizationalgorithm.

The measurement covariance matrix is determined basedon the accuracy of the measurements. The appropriatechoice of the state covariance matrix, Q, is however oftendifficult in practical applications. An estimate of Q can beobtained by assuming that the process noise vector mostlyrepresents the effects of parametric uncertainty (Valappil &Georgakis, 2000). Based on this assumption and perform-ing a first-order power series expansion of the model errorequations using the nominal parameter vector and controltrajectory, the process noise covariance matrix can becomputed as follows:

QðtÞ ¼ SyðtÞVySTy ðtÞ, (24)

where Vy 2 Rny�ny is the parameter covariance matrix, andSyðtÞ is the jacobian computed using the nominal para-meters and estimated states:

SyðtÞ ¼@f

@y

� �xðtÞ;uðtÞ;y

. (25)

Eq. (24) provides an easily implementable way toestimate the process noise covariance matrix, since theparameter covariance matrix Vy is usually available fromparameter estimation, and the sensitivity coefficients inSy(t) can be computed by finite differences or via sensitivityequations (Feehery, Tolsma, & Barton, 1997). Note thatthe above approach leads to a time-varying, full covariancematrix, which has been shown to provide better estimationperformance for batch processes than the classically usedconstant, diagonal Q (Nagy & Braatz, 2003a, b; Valappil &Georgakis, 2000).

3. Setpoint tracking NMPC of the industrial batch reactor

A schematic representation of the experimental pilotplant is shown in Fig. 2. The reactor temperature iscontrolled using a complex heating–cooling system, whichis based on a closed oil circuit, which is recycled throughthe jacket with a constant flow rate Fj. The heating–coolingmedium goes through a multi-tubular heat exchangerwhere a PI controller adjust the cooling water flow rate

ARTICLE IN PRESS

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

time

Mn,

Mw

Mn modelMn experimentalMw modelMw experimental

Fig. 3. Comparison of experimental data to the kinetic simulation model.

The number average and weight average molecular weights (Mn, Mw) are

in normalized values.

Z.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850844

with the aim to keep the temperature difference betweenthe input and output of the heat exchanger constant.Heating is performed using an electric heater. The power ofthe heater is adjusted by the lower level PI controller thatcontrols the input temperature into the jacket. The setpointof the PI controller is determined by the higher levelNMPC that has the objective to track a predeterminedtemperature profile in the reactor.

3.1. Optimization, control relevant modelling and

identification of the reactor

A detailed first-principles model of the process contain-ing material and energy balances as well as detailed kineticand thermodynamic models was developed previously ingPROMS (an advanced dynamic modelling and simulationenvironment) for another large scale reactor with differentheating/cooling system, and was available in the companyfor the controller design. Based on this model a simplifiedfirst-principles model was developed as a Simulink S-function in Matlab and C, and was used in the developedNMPC software environment. The simplified model keepsthe core structure of the generic kinetic model based on thefree radical polymerization mechanism (Penlidis, Ponnus-wamy, Kiparissides, & O’Driscoll, 1992), implemented inthe original complex model, to which energy balances andlower level controller equations were added to accommo-date the architecture of the heating/cooling system at thepilot plant used for the experiments. The simplifiedrepresentation of the kinetic model is given in terms ofthe low order moments of both the live and dead polymerchain distributions. Defining Ri and Di as the molarconcentration of the free-radical and the dead polymerspecies, respectively the general kth moments are:

lk ¼X1i¼1

ikRi, (26)

mk ¼X1i¼1

ikDi, (27)

with l ¼ ½l0; l1; l2�T and m ¼ ½m0;m1;m2�T. The material

balances expressed in terms of the moments lead to thefollowing system of DAEs:

_Z1 ¼ f 1ðZ1; y1Þ, (28)

g1ðl;Z1; y1Þ ¼ 0, (29)

with Z1 ¼ ½nM ; I1; I2; ðmkV Þ�T, with nM is the monomerconcentration (number of moles), I1 and I2 are the twoinitiator concentrations, V is the time-varying volume ofthe reaction mixture, and y1 a parameter vector containingthe kinetic parameters. The initial condition isZ1ð0Þ ¼ ½nM;0; I1;0; I2;0; 0; 0; 0�

T, with nm,0, I1,0, and I2,0 theinitial monomer and initiator concentrations, respectively.The quasi-steady-state assumption was used for theconcentration of the free radical polymers leading to the

algebraic Eq. (29). An additional set of equations were usedto incorporate the energy balances and the effect of thelower-level controllers in the model. A series of empiricalalgebraic equations were also included in the complexmodel together with (28)–(29) to model the gel effect andthe temperature variation of physico-chemical properties.The number average molecular weight and the weight

average molecular weight are computed by

Mn ¼MWm1ðtÞm0ðtÞ

, (30)

Mw ¼MWm2ðtÞm1ðtÞ

, (31)

where MW is the molecular weight of the monomer.Generally the two parameters Mn and Mw are in directcorrelation with the quality of the polymer and are stronglydependent on the temperature profile of the polymerizationand ultimately on the performance of the trackingcontroller. The initial kinetic parameters (from literatureand previous experiments) were verified and retuned usinglaboratory data (Mn,Mw and conversion measurements)collected during the NMPC experiments on the pilotreactor, and model (28)–(29), without the energy balances,using the measured time-varying reactor temperature in theparameter tuning. The good fit of the model to theexperimental data is shown in Figs. 3 and 4. Detailedopen-loop optimization studies were also performed off-line, to determine the optimal temperature profile in thereactor and compare it to the previously used setpoint fromthe a bechmark recipe. The rigorous model was also used insimulation experiments to develop a two layer NMPC, thesimultaneously solves the endpoint and tracking problem(Nagy et al., 2004). The detailed and retuned S-functionmodel was used to solve the following optimal controlproblem, with the objective of minimizing the batch timewhile satisfying certain quality constraints (to keep the

ARTICLE IN PRESS

0

0.2

0.4

0.6

0.8

1

time

Con

vers

ion

Model

Experimental

Fig. 4. Comparison of experimental conversion data to the values from

the kinetic simulation model.

time

Tem

pera

ture

N = 5

N = 10

Fig. 5. Optimal temperature profile for the batch polymerization reactor

using two different discretization levels.

Z.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850 845

average molecular weights within desired limits), as well asachieving a minimum conversion required from productiv-ity and safety considerations:

minTrðtÞ

tf (32)

subject to:

Mw;minðtf ÞpMwðtf ÞpMw;maxðtf Þ, (33)

Mn;minðtf ÞpMnðtf ÞpMn;maxðtf Þ, (34)

xminðtf Þpxðtf Þ, (35)

Tr;minðtÞpTrðtÞpTr;maxðtÞ, (36)

_Tr;minðtÞp _TrðtÞp _Tr;maxðtÞ, (37)

where the first two constraints guarantee that the productwill be within the box constraint given by qualityspecification at the end of the batch, the third constraintaccounts for achieving the desired conversion xminðtf Þ at theend of the batch, whereas the last two constraints makesure that the resulted temperature profile is within theoperating range. The infinite-dimensional optimizationproblem (32)–(37), is solved by discretizing the reactortemperatures in a piecewise linear vector function with afinite number of elements N. The resulting finite-dimen-sional optimization problem was solved using the multipleshooting approach in a similar way as described in Section2.3. Simulation studies show that in this particular examplethe number of discretization does not have a verysignificant effect on the final result, as long as NX3.Fig. 5 shows the optimization results for N ¼ 5 and 10. Theoptimal setpoint profile reduces the batch time with morethan 5%min compared to the initial recipe. Using thelarger discretization slightly shorter batch time can beobtained. It was observed that independently of the level ofdiscretization the optimal profiles seem to have three

stages. The first is determined by the maximum heating rateallowed in system (37) and the last by the maximumtemperature in reactor (36), with a very short second periodwhen constraints (36) or (37) are not active, whichcorresponds to the initial part of the polymerizationreaction. This corresponds to kinetic studies published inthe literature showing that the molecular size distribution ismainly determined in the initial part of the polymerization,whereas the second part is to achieve desired conversion. Inthe NMPC study a temperature profile consisting of N ¼ 3linear segments was used.Since only temperature measurements are available in

the plant, many states of the detailed model are notestimable, or not even detectable. The complex modelhowever was used to determine the optimal temperatureprofile, and for deriving the control-relevant model,according to the procedure shown in Fig. 6. Based on thistwo-step procedure, first a complex mechanistic model isderived, as described above, which is then used to obtainthe control-relevant model, so that particular systemstheoretical properties are achieved (e.g. controllability orobservability), as well as the computational complexity ismanageable to guarantee real-time feasibility. In thisapproach one part of the equations from the complexmechanistic model is kept or mapped in a new set ofsimplified first-principles equations, whereas the rest of themodel equations are condensed in an empirical nonlinearfunction, so that the resulted grey-box model is observable.Unlike the linear case, for nonlinear models there is nogeneric algorithm to derive the control relevant model fromthe complex first-principle model. This step involvesengineering heuristics and detailed inside knowledge aboutthe models. In the presented study comprehensive obser-vability analysis was performed to obtain the structure ofthe control-relevant model.In the experimental pilot plant the available measure-

ments are: reactor temperature (Tr), and input and output

ARTICLE IN PRESS

x1,x2,K,xm,xm+1,K,xn

Detailed first-principles model ⇒ Open-loop optimal control

Nonlinear map

Y=M (x1,x

2,K,x

n) x

m+1,K,x

n,Y

Reduced first-principles model

Control-relevant (grey-box) model

Setpoint

⇒ Closed-loopNMPC

Off-line

On-line

Fig. 6. Schematic representation of the two-steps approach to derive the

control relevant model from a detailed first principles model. The control

relevant model consists of a subset of the equations from the detailed first-

principles model (or a mapped set of equations via additional simplifying

assumptions) and an empirical nonlinear map, so that the resulted hybrid

model is observable, and guarantees real-time feasibility of the on-line

optimization.

0 5 10 15 20time (h)

Rea

ctor

tem

pera

ture

SimulationExperiment

Fig. 7. Validation of the model compared to plant data.

Z.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850846

temperatures into and from the jacket, (Tjin,Tj). With thisset of measurements the following reduced model was usedin the NMPC:

_nM ¼ �Qr=DHr, (38)

_Tr;k ¼ ðQr þUwAwðTw;k � Tr;kÞ � ðUAÞloss;rðTr;k � TambÞÞ

=ðmMcp;M þmPcp;P þmwatercp;waterÞ;

(39)

_Tw;k ¼ ðUjAjðTj;k � Tw;kÞ �UwAwðTw;k � Tr;kÞÞ=mw=cpw,

(40)

_Tj;k ¼ ðNFjrjcp;jðTj;k�1 � Tj;kÞ �UjAjðTj;k � Tw;kÞ

�ðUAÞloss;jðTj;k � TambÞÞ=ðmjcp;jÞ;(41)

where k ¼ 1; . . . ;N, Tr ¼ Tr;N, Tj ¼ Tj;N, Tj,0 ¼ Tj,in, nM

is the number of mol of monomer, DHr is the enthalpy ofreaction, Tw is the wall temperature, U and A are heattransfer coefficients and areas from reactor to wall ð�Þw orwall to jacket ð�Þj, cp;M=P=water=w=j and mM=P=water=w=j are theheat capacities and masses of monomer, polymer, water,wall and oil, Tamb is the ambient temperature, rj is thedensity of the oil, ðUAÞloss;r=j heat loss coefficients in thereactor and jacket, respectively.

To estimate the transport delay, the reactor, wall andjacket were divided in N ¼ 4 elements, leading to a systemof 13 differential equations. To achieve proper predictionand maintain the observability of the model, with onlytemperature measurements available, different approacheshave been proposed. Helbig, Abel, M’hamdi, & Mar-quardt, (1996) used a time series of the estimated heatgeneration determined from simulation of a batch. Theindustrial batch MPC product developed by IPCOS deter-mines an empirical nonlinear relation Qr ¼ f QðnM ;TrÞ,which expresses the heat generation as a function of theconversion and temperature (Van Overschee & VanBrempt, 2000). In our case-study a similar approach wasused. The empirical nonlinear relation was determinedfrom the complex first-principle model, simulating the

process for different temperature profiles. Differentapproaches have been tried to obtain the empirical modelfQ, radial basis neural networks and sum of Gaussiansapproximations gave the best fit with the smallest numberof parameters providing practically identical results inestimating the Qr ¼ f QðnM ;TrÞ generated from the first-principles model.Maximum likelihood estimation was used to fit the

parameters of model (38)–(41) to the data obtained fromthe plant, performing several water batches (when Qr ¼ 0),and minimizing the objective function below:

cMLEðyÞ ¼ ðnexp=2ÞXnmeas

i¼1

ln ðXnexpj¼1

�ijðyÞÞ (42)

where nexp is the number of experimental points, from eachof the nmeas sensors, and eij is the error between themeasurement and model prediction. The following para-meter vector was used in the estimation:

y00 ¼ ½ðUAÞloss;r; ðUAÞloss;j ; UjAj ; mw; mj �. (43)

This procedure gives the optimal nominal parameterestimates, y

, and the corresponding uncertainty descrip-

tion given by the covariance matrix, estimated fromHessian of the objective (42) at the optimal parameterestimate, Vy H�1 ¼ ð@2c=@y2Þ�1

y¼y . The good fit between

the experimental data and the model is shown in Fig. 7.

3.2. NMPC of the bath reactor

Model (38)–(41) was used in an adaptive outputfeedback NMPC approach, where the objective was toprovide a tight setpoint tracking, by minimizing online, inevery sampling instance k, the following quadratic objec-tive:

minuðtÞ

Z tF

tk

fðTrðtÞ � Tr;SPðtÞÞ2þQDuðduðtÞÞ2gdt. (44)

ARTICLE IN PRESS

time

Tem

pera

ture

Tj,SP= u(t) from NMPC

Tj,plant

Tj,SP,predicted

Fig. 9. NMPC control input profiles (Tj,SP,predicted), the finally implemen-

ted control input (Tj,SP), and the real jacket temperature Tj,plant.

Z.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850 847

The setpoint profile Tr,sp used in the controller was givenby a previously used benchmark recipe as described in theprevious section. The manipulated input of the NMPC,uðtÞ ¼ Tj;SP, is the setpoint temperature to the lower levelPI controller, which controls the input temperature into thejacket. Real-time simulations were performed when thecomplex model was used to simulate the real plant on onecomputer, whereas the NMPC was running on a differentcomputer. The communication between the plant andNMPC was performed via the standard OPC interface,simulating the real industrial setup. The performance of theNMPC is presented in Figs. 8 and 9. The grey bands in thefigures represent the set of model predictions and controlinputs during the batch. Fig. 8 demonstrates a goodsetpoint tracking performance of the controller andprediction abilities of the model, which has been adaptedduring the batch. The larger control error during the firsthalf of the constant temperature profile (or the end ofthe batch) is caused by the saturation of the lower levelPI controller, which can also be observed on Fig. 9. Thelarger band formed by the whole NMPC solutions(Tj,SP,predicted) computed in each sampling time shows thatin those specific periods of the batch the setpoint valuesgiven by the NMPC cannot be achieved by the lower-levelPI controller. This error can be eliminated only if thelower-level PI controller is introduced in the predictionmodel, described by an additional set of two differentialequations:

_uPI ;0 ¼ 0, (45)

_Ie ¼ �ðtÞ, (46)

where uPI is the low-level control input (power to theelectric heater in percentage), uPI,0 is the control input att ¼ 0, �ðtÞ ¼ Tj;SP � Tj;in is the control error, IeðtÞ ¼

R t

0 �ðtÞ

time

Tem

pera

ture

Tr,SP

Tr

Tr,predicted

Fig. 8. Setpoint tracking performance of the NMPC, and model

predictions during the batch.

is the integral error in the typical PI controller equation:

uPI ðtÞ ¼ uPI ;0ðtÞ þ KPI ð�ðtÞ þ1

Ti;PI

IeðtÞÞ, (47)

where KPI and Ti,PI are the tuning parameters of the PIcontroller, and including Eqs. (45)–(47) with (38)–(41) inthe predication model, and adding the constraints0puPI ðtÞp100 to the NMPC optimization problem thesaturation of the low-level controller can be avoided.Similar conclusions have been achieved from the

experiments conducted in the real pilot plant located atBASF in Ludwigshafen, Germany. The experimentsdemonstrate the improvement in the control performanceif the lower level controller is modelled (in scenarios whensaturation can occur) and illustrate the necessity of properonline model adaptation. The experimental results from thepilot plant implementation of the NMPC are presented inFig. 10. The NMPC controller is able to track totemperature profile within 70.5 1C. Fig. 11 shows thatthe new NMPC formulation is able to avoid controllersaturation, indicated by consistency of the whole NMPCcontrol inputs computed in each iteration, leading to thenarrow band on the plot.In the implementations the parameters y0 ¼ ½Qr; UwAw�

were estimated together with the model states in thePAEKF. The parameter covariance matrix Vy, resultedfrom the identification was used to compute the statecovariance matrix in the estimator according to (24). Thisapproach provides a good estimate of the generated heat,shown in Fig. 12 and the heat transfer properties (Fig. 13).Note that the heat transfer coefficient (Uw Aw) variessignificantly along the batch, thus the adaptation of themodel through the estimated parameter is very important.The variation of (Uw Aw) observed from the estimation is incorrelation with the real physical phenomena, explained bythe variation of the viscosity of the reaction mixture, which

ARTICLE IN PRESS

time

Tem

pera

ture

-1.5

-1

-0.5

0

0.5

1

1.5

Con

trol

Err

or (

°C)

Tr

Tr,SP

Fig. 10. Experimental data with the setpoint tracking performance of the

NMPC in the BASF pilot plant. The reactor temperature almost perfectly

follows the setpoint (within 70.5 1C).

time

Tem

pera

ture

Tj,SP - u(t) from NMPC

Tj,plant

Tj,SP,predicted

Fig. 11. Experimental data from the pilot pant, with the NMPC control

input profiles (Tj,SP,predicted), the finally implemented control input (Tj,SP),

and the real jacket temperature Tj,plant.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

time

Gen

erat

ed h

eat

ModelEstimated

Fig. 12. Normalized estimated and real (obtained with the detailed kinetic

model) reaction heat (Qr) during the experimental NMPC run.

0

100

200

300

400

500

600

700

time

Uw

Aw

(W

/K)

Fig. 13. Estimated heat transfer coefficient along the batch.

0

5

10

15

20

25

time

CP

U (

s)

Sampling time

Fig. 14. Total CPU times (estimation+optimization) along the batch.

Z.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850848

increases significantly during the batch. Additionally, thisparameter is also a ‘‘tuning’’ knob of the model andovertakes the effects of model plant mismatch for offset-free state estimation, therefore part of the variation in(UwAw) is due to model adaptation. A weighting coefficientof QDu ¼ 0.4, and prediction and control horizons of 4were used, in the optimization, with a sampling time of20 s. The control input was discretized in 400 piecewiseconstant inputs, leading to a significantly high dimensionaloptimization problem. The efficient multiple shootingapproach guarantees the real-time feasibility of the NMPCimplementation. Even with the large control discretizationof 400 piece-wise linear segments the computation time wasbelow the sampling time of 20 s, as can be seen in Fig. 14.

To illustrate the importance of accurate modelling andpredication in NMPC applications in one experiment the

lower-level PI controller model and the heat generationfunction Qr ¼ f QðnM ;TrÞ were switched off before thesecond heat generation peak occurred in the reactionaccording to Fig. 12. The PAEKF adapted the model andestimated the generated heat, but in the prediction constantvalues (current estimate) were considered for the entireprediction horizon. This implementation led to saturation

ARTICLE IN PRESS

time

Tem

pera

ture

-2

-1

0

1

2

3

4

Con

trol

Err

or (

°C)

Tr

Tr,SP

Coagulationstarted

Prediction of Qrswitched off

Fig. 15. Experimental results with the NMPC in the scenario when

prediction of heat generation and slave controller model were switched off

before the second peak of generated heat (see Fig. 12) leading to

coagulation and compromised batch.

time

Tem

pera

ture

(°C

)

-1.5

0

1.5

3

4.5

6

7.5

Con

trol

Err

or (

°C)

Tr

Tr,SP

Fig. 16. Experimental results with cascade PI control of the pilot plant

reactor (sampling time of 3 s).

time

Tem

pera

ture

-4

-3

-2

-1

0

1

2

3

4

Con

trol

Err

or (

°C)

Tr

Tr,SP

Fig. 17. Simulation results using linear model predictive control, with

measurement noise of 70.05 1C (based on experimental observations),

same sampling time of 5 s and same prediction and control horizon as for

the NMPC.

Table 1

Integral square errors and maximum error for the three control algorithms

Controller ISE Max|Tsp�Tr

NMPC 0.12 0.6

Cascade PI 16.34 4.4

Linear MPC 9.86 3.1

Z.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850 849

of the lower level controller and significantly worse controlperformance shown in Fig. 15. Ultimately in this experi-ment the coagulation of the product has also occurredleading to compromised batch, which might only beavoided by including even more details into the model byincorporating the effect of coagulant on the particle sizedistribution and using additional sensors. In this setup theappropriate moment to start the coagulant introductionwould be triggered by the NMPC prediction and the rate ofintroduction could be a second NMPC control variable,which would be used to adjust the important macroscopicproperties of the product related to the particle sizedistribution. During the first half of the batch, when theNMPC was used with the appropriate process model thatincludes lower PI controllers and prediction of Qr, thecontrol performance is very good, despite the large and fastvariation in the heat generation, showing the importance ofproper control-relevant model in the NMPC.

The performance of the NMPC controller was comparedexperimentally to the classical cascade PI control, shown inFig. 16, and to linear MPC (Fig. 17) using the real-timesimulator setup and a linear model identified in theoperating point at the middle of the first linear segmentof the setpoint profile. The control performance quantifiedby the integral square error (ISE) and the maximum errorare given in Table 1 for the three cases (Figs. 10, 16and 17). It can be observed that the NMPC providessuperior performance. The available PI cascade system hasdifficulty in following the ramp temperature profile andleads to large control error after the maximum heatgeneration peak. As the reaction heat increases the PIcontroller increases the cooling power in the system, andthe sudden decrease in reaction heat unforeseen by thecontroller leads to overcooling. The linear MPC, due to itspredictive nature provides better performance than the PIcontrol, avoiding the large temperature deviations due to

reaction heat peaks, but show significantly more oscillatorybehaviour. Although NMPC has provided clearly bettertracking performance than the cascade PID and linearMPC, the lower temperature sensitivity of this particularprocess made the implementation effort associated toNMPC implementation economically questionable andthe cascade PID control provides acceptable performancefrom the practical point of view. However, the advanced

ARTICLE IN PRESSZ.K. Nagy et al. / Control Engineering Practice 15 (2007) 839–850850

NMPC strategy would allow further reduction of batchtime by designing more aggressive setpoint profiles, orusing more detailed model (including the effect ofcoagulation) in the controller the predictive nature of theNMPC can reduce the number of compromised batchesdue to agglomeration.

4. Conclusions

The paper presents a computationally efficient NMPCapproach and tool that combines output feedback designwith efficient optimization technique to provide a nonlinearmodel predictive control framework that can be supportedin an industrial environment. Detailed first-principlesmodel is used to derive the reduced control-relevant modelbased on the available measurements. The model is tunedusing data from the plant, and used then in the NMPC. APAEKF is combined with the control algorithm for the on-line state estimation and model adaptation to achieve offsetfree control. Simulation and experimental results demon-strate the efficiency of the NMPC approach in an industrialpilot plant application, in comparison to classical cascadeand linear MPC. Although the final conclusion of theevaluation study is that the application of NMPC for thisparticular process is not economically justifiable, theimportance of proper modelling aspects are also illustratedin simulations and experimental results. Additionally theapproach presented here is easily applicable to other batchprocesses, where higher sensitivity of the product qualityon the temperature variation, or multi-variable nature (e.g.for semibatch systems) would take more advantage of theproposed approach. The NMPC tool described can be usedfor rapid prototyping and evaluation of the algorithm forany process in an industrial environment, making it a goodmeans to broaden the acceptance of this technique in theindustrial community.

Acknowledgement

The authors thank Mr. Heinrich Liede-Burgemeister(BASF) and Dr. J. Gao (BASF) for the analyticalmeasurements used for the verification and tuning of thekinetic model.

References

Biegler, L. (2000). Efficient solution of dynamic optimization and NMPC

problems. In F. Allgower, & A. Zheng (Eds.), Nonlinear predictive

control. Bagel: Birkhauser.

Bequette, B. W. (1991). Nonlinear control of chemical processes—A

review. Industrial Engineering Chemical Research, 30, 1391–1413.

Bodizs, L., Titica, M., Faria, N., Srinivasan, B., Dochain, D., & Bonvin,

D. (2006). Industrial experience with a fed-batch filamentous fungal

fermentation: modeling and oxygen control. Journal of Process

Control, in press, 2006.

Bonvin, D. (1998). Optimal operation of batch reactors: A personal view.

Journal of Process Control, 8, 355–368.

Bonvin, D., Srinivasan, B., & Hunkeler, D. (2006). Performance

improvement in batch processes: control and optimization challenges

in the production of specialty chemicals, IEEE Control Systems

Magazine, in press, 2006.

Diehl, M. (2001). Real-time optimization for large scale nonlinear processes.

Ph.D. thesis, University of Heidelberg.

Diehl, M., Bock, H. G., Schloder, J. P., Findeisen, R., Nagy, Z., &

Allgower, F. (2002). Real-time optimization and nonlinear model

predictive control of processes governed by differential–algebraic

equations. Journal of Process Control, 12, 577–585.

Feehery, W. F., Tolsma, J. E., & Barton, P. I. (1997). Efficient sensitivity

analysis of large-scale differential-algebraic systems. Applied Numerical

Mathematics, 25, 41–54.

Franke, R., Arnold, E., & Linke, H. HQP: A solver for nonlinearly con-

strained large-scale optimization. http://www.sourceforge.nethttp://

hqp.sourceforge.net.

Helbig, A., Abel, O., M’hamdi, A., & Marquardt, A.W. (1996). Analysis

and nonlinear model predictive control of the Chylla-Haase Bench-

mark problem. Proceedings of UKACC International conference on

Control, pp. 1172–1177.

Henson, A. M. (1998). Nonlinear model predictive control: Current status

and future directions. Compters and Chemical Engineering, 23,

187–201.

Li, S., Petzold, L. (1999). Design of new DASPK for sensitivity analysis.

UCSB Technical report.

Morari, M., & Lee, J.H. (1997). Model predictive control: Past, present

and future. In Proceedings of PSE’97-ESCAPE-7 symposium, Trond-

heim.

Nagy, Z.K., Allgower, F., Franke, R., Frick, A., & Mahn, B. (2004).

Efficient tool for nonlinear model predictive control of batch

processes. In Proceedings of the 12th Mediterranean Conference on

Cont. and Aut. (MED’04), Kusadasi, Turkey.

Nagy, Z. K., & Braatz, R. D. (2003a). Robust nonlinear model predictive

control of batch processes. AIChE Journal, 49, 1776–1786.

Nagy, Z.K., & Braatz, R.D. (2003b). Recent advances in the optimal

control of batch processes. In: Recent research developments in

chemical engineering.Kerala: Vol. 5 TransWorld Research Network,

Kerala, 5, pp. 99–127.

Penlidis, A., Ponnuswamy, S. R., Kiparissides, C., & O’Driscoll, K. F.

(1992). Polymer reaction engineering: Modelling considerations for

control studies. The Chemical Engineering Journal, 50, 95–107.

Qin, S. J., & Badgewell, T. (2003). A survey of industrial model predictive

control technology. Control Engineering Practice, 11, 733–764.

Srinivasan, B., Bonvin, D., Visser, E., & Palanki, S. (2003). Dynamic

optimization of batch processes: II. role of measurements in handling

uncertainty. Computers and Chemical Engineering, 27, 27–44.

Valappil, J., & Georgakis, C. (2000). Systematic estimation of state noise

statistics for extended Kalman Filters. AIChE Journal, 46, 292–308.

Van Overschee, P., & Van Brempt, W. (2000). Polyproms IPCOS-ISMC

final report, internal report of the polyPROMS European 5th

framework research project GRD1-2000-25555.

Welz, C., Srinivasan, B., & Bonvin, D. (2006). Measurement-based

optimization of batch processes: meeting terminal constraints via

trajectory following. Journal of Process Control, in press.