Qup

JD

a

ARAA

KNMNNQ

1

ppaiabtefi

0h

Journal of Process Control 24 (2014) 57–71

Contents lists available at ScienceDirect

Journal of Process Control

journa l homepage: www.e lsev ier .com/ locate / jprocont

uality relevant nonlinear batch process performance monitoringsing a kernel based multiway non-Gaussian latent subspacerojection approach

unichi Mori, Jie Yu ∗

epartment of Chemical Engineering, McMaster University, Hamilton, Ontario, Canada L8S 4L7

r t i c l e i n f o

rticle history:eceived 3 September 2013ccepted 28 October 2013vailable online 18 December 2013

eywords:onlinear batch processultidimensional mutual informationonlinear kernel feature spaceon-Gaussian latent subspace projectionuality relevant batch process monitoring

a b s t r a c t

Multiway kernel partial least squares method (MKPLS) has recently been developed for monitoring theoperational performance of nonlinear batch or semi-batch processes. It has strong capability to handlebatch trajectories and nonlinear process dynamics, which cannot be effectively dealt with by traditionalmultiway partial least squares (MPLS) technique. However, MKPLS method may not be effective in captur-ing significant non-Gaussian features of batch processes because only the second-order statistics insteadof higher-order statistics are taken into account in the underlying model. On the other hand, multiway ker-nel independent component analysis (MKICA) has been proposed for nonlinear batch process monitoringand fault detection. Different from MKPLS, MKICA can extract not only nonlinear but also non-Gaussianfeatures through maximizing the higher-order statistic of negentropy instead of second-order statisticof covariance within the high-dimensional kernel space. Nevertheless, MKICA based process monitoringapproaches may not be well suited in many batch processes because only process measurement vari-ables are utilized while quality variables are not considered in the multivariate models. In this paper,a novel multiway kernel based quality relevant non-Gaussian latent subspace projection (MKQNGLSP)approach is proposed in order to monitor the operational performance of batch processes with nonlinearand non-Gaussian dynamics by combining measurement and quality variables. First, both process mea-surement and quality variables are projected onto high-dimensional nonlinear kernel feature spaces,respectively. Then, the multidimensional latent directions within kernel feature subspaces correspond-

ing to measurement and quality variables are concurrently searched for so that the maximized mutualinformation between the measurement and quality spaces is obtained. The I2 and SPE monitoring indiceswithin the extracted latent subspaces are further defined to capture batch process faults resulting inabnormal product quality. The proposed MKQNGLSP method is applied to a fed-batch penicillin fermen-tation process and the operational performance monitoring results demonstrate the superiority of thedeveloped method as apposed to the MKPLS based process monitoring approach.

. Introduction

Batch and semi-batch processes have been widely used toroduce low-volume but high-value-added products in polymer,harmaceutical, semiconductor, materials, food, biotechnology,nd agricultural industries. In order to improve product qual-ty, plant safety, energy efficiency, environmental sustainabilitynd economic profit, effective process performance monitoring isecoming critically important and have received significant atten-

ion in academia and industry. During batch process operation,ven small process variations may have substantial impact on thenal product quality, yields, production efficiency, environmental

∗ Corresponding author. Tel.: +1 9055259140x27702; fax: +1 9055211350.E-mail address: jieyu@mcmaster.ca (J. Yu).

959-1524/$ – see front matter © 2013 Elsevier Ltd. All rights reserved.ttp://dx.doi.org/10.1016/j.jprocont.2013.10.017

© 2013 Elsevier Ltd. All rights reserved.

sustainability, etc. While it is possible to detect defective productsin batch or semi-batch processes by tracking final product qual-ity, it cannot prevent the abnormal operating events and avoid thewasted batches. Therefore, it is highly desirable to conduct on-line monitoring throughout batch operation so as to detect andresolve process faults in early stage [50,9,49,43]. On the other hand,nowadays most industrial plants are fully instrumented with largenumber of sensors and analyzers, which enable data-driven processperformance monitoring and diagnosis.

The traditional process monitoring, fault detection and diagno-sis methods are based on first-principle process models. However,the reliability of monitoring approaches heavily depends on the

accuracy of the mechanistic models, which require in-depth pro-cess knowledge and analysis. Moreover, it can be very tediousand time-consuming to develop mechanistic models of complexprocesses [33,3,13]. Alternately, multivariate statistical process

5 rocess

mctat[pboalbuttompttpTsbcMea

pP[Kc(cqDsvsncetott

es[ottndncktcntis

8 J. Mori, J. Yu / Journal of P

onitoring (MSPM) techniques have been widely applied to bothontinuous and batch processes with many successful applica-ions. Typical MSPM approaches can capture variable correlationsnd then transform the original process measurement space intohe latent-variable subspace based on historical operating data32,30,35,8,31,38,34,39–41]. Two popular MSPM methods, princi-al component analysis (PCA) and partial least squares (PLS), haveeen intensively used to monitor and diagnose the performancef different types of processes. Both methods can handle vari-ble collinearity and project process data onto lower-dimensionalatent subspace that retains most of the operational informationased on variance or covariance. Then the T2 and SPE indices aretilized to isolate abnormal events from normal process opera-ion [20,26,2,7,54]. Nevertheless, batch process data typically havehree-dimensional structure while the regular PCA and PLS meth-ds can only handle two-dimensional data matrices. Therefore,ultiway principal component analysis (MPCA) and multiway

artial least squares (MPLS) techniques are developed to handlehree-dimensional batch process data [30,42,27,14]. Furthermore,he dynamic model-based process monitoring technique is pro-osed to characterize variable dynamics of batch processes [10].his method filters process data through a multivariate autoregres-ive model before performing PCA so as to capture the dynamicehavior of process variables. However, batch and semi-batch pro-esses are often characterized with strong nonlinearity so thatPCA/MPLS based process monitoring approaches may not be

ffective in detecting abnormal operations because both methodsre essentially based on multivariate linear models [51,44].

In order to tackle process nonlinearity, kernel principal com-onent analysis (KPCA) is developed as an extension of regularCA method for nonlinear process monitoring and diagnosis21,22,25,28]. The high-dimensional kernel feature space enablesPCA method with the capability of extracting nonlinear processharacteristics. Furthermore, multiway kernel partial least squaresMKPLS) approach is developed for batch process monitoring byapturing the covariance structure between measurement anduality variables in high-dimensional kernel feature space [12].ifferent from KPCA, KPLS has the inherent input–output model

tructure through combining process measurement and qualityariables. Nevertheless, KPCA and KPLS techniques rely only onecond-order statistics within kernel feature space and thus mayot effectively extract all non-Gaussian features from batch pro-esses that are characterized by higher-order statistics such asntropy and mutual information. Meanwhile, the validity of the sta-istical confidence limits of T2 and SPE indices in PLS model reliesn the assumption that the process data follow Gaussian distribu-ion approximately. Such requirement may not be satisfied whenhe normal operating data are of significant non-Gaussianity.

Alternately, independent component analysis (ICA) method ismployed to project multivariate process data onto latent sub-pace consisting of statistically independent components (IC).18,1,23,6,11,15,17]. The basic idea of ICA is to maximize the higher-rder statistics such as negentropy, which can not only de-correlatehe process data but also reduce statistical dependencies amonghe extracted latent variables. Thus, ICA can effectively captureon-Gaussian process features [24]. Moreover, kernel indepen-ent component analysis (KICA) method is developed to deal withonlinear dynamics for process monitoring [53]. First, principalomponents are computed from process data in high-dimensionalernel feature space by performing KPCA. Then, ICA is utilizedo search for the latent directions within the kernel-principal-omponents subspace so as to identify process nonlinearity and

on-Gaussianity. KICA can be integrated with multiway analysiso handle three-dimensional data set for batch process monitor-ng and fault detection [36]. In addition, the kernel independentcores from KICA model can be directly utilized as the inputs of

Control 24 (2014) 57–71

support vector machine (SVM) in order to diagnosis process faults[52]. However, the above ICA based techniques may not be appro-priate for quality-relevant batch process monitoring because theunderlying ICA models do not incorporate quality variables asoutputs. Thus, the detected abnormal operating events may notcause any product quality degradations during batch operationand can trigger many unnecessary fault alarms in the monitor-ing systems. More recently, multiway Gaussian mixture model(MGMM) is developed to characterize non-Gaussian features andmonitor operational performance of multi-phase batch processes[48,55]. Gaussian mixture model consists of multiple Gaussiancomponents corresponding to different operating phases through-out batch operation and can be applied to batch processes withmultiple phases and shifting dynamics [46,47]. Furthermore, theGaussian mixture model is extended to nonlinear kernel featurespace for monitoring complex processes that involve nonlineardynamics within each local operating mode or phase [45]. Nev-ertheless, the above GMM based process monitoring approachesstill do not take into consideration product quality variables andthereby they may not specifically identify abnormal operatingevents that cause deteriorated product quality throughout batchoperation.

In this study, a novel multiway kernel based quality relevantnon-Gaussian latent subspace projection (MKQNGLSP) method isdeveloped to monitor quality related operational performance ofnonlinear batch processes. First, three-dimensional batch processdata are unfolded into two-dimensional matrices through multi-way analysis for handling batch trajectories. Then, kernel principalcomponents of measurement and quality variables are extractedto form high-dimensional feature spaces, respectively. The non-Gaussian multidimensional latent directions corresponding tomeasurement and quality variables within kernel-principal-component subspaces are further estimated through maximizingmutual information between measurement and quality variables.Thus, the MKQNGLSP based SPE and I2 indices along with the cor-responding confidence limits are derived to capture non-Gaussianprocess abnormalities of nonlinear batch operation. Different fromMKPLS, the presented MKQNGLSP approach is based upon thehigh-order statistic of mutual information instead of the second-order statistic of covariance so that the non-Gaussian relationshipsbetween measurement and quality variables can be well identified.Moreover, the MKQNGLSP model includes measurement and qual-ity variables so that the quality relevant process monitoring canbe conducted to detect faults with significant influences on finalproduct quality.

The organization of the rest of the article is as follows. Sec-tion 2 briefly reviews the multiway KPCA (MKPCA), multiwayKICA (MKICA), MKPLS based batch process monitoring techniques.Section 3 describes the proposed multiway kernel based qualityrelevant non-Gaussian latent subspace projection approach formonitoring nonlinear batch processes. In 4, the presented methodis applied to the example of fed-batch penicillin fermentation pro-cess and the superiority of the MKQNGLSP method as apposed toMKPCA, MKICA and MKPLS approaches is demonstrated. Finally,the conclusions are provided in Section 5.

2. Preliminary

KPCA, KICA and KPLS methods can deal with the nonlinearlycorrelated multi-dimensional data through the kernel projec-tion of measurement data onto the high-dimensional feature

space. While the purpose of KPCA and KICA is to maximize thesecond-order statistic of variance and higher-order statistic ofnegentropy among the projected measurement variables, respec-tively, KPLS aims to obtain the maximized covariance between

rocess

tsatsKpcpXtctfiaYm

2

a

�

wt

C

wcet

K

wcR

K

wsf

t

wR

fup

T

S

wlt

J. Mori, J. Yu / Journal of P

he projected measurement and quality variables. Given b batches,sampling instants for each batch, vm measurement variables,

nd vq quality variables, batch process data can be expressed ashree-dimensional matrices X ∈ Rb×vm×s and Y ∈ Rb×vq×s for mea-urement and quality variables, respectively. Since KPCA, KICA andPLS can deal with two-dimensional data matrices only, the batchrocess data need to be unfolded into two-dimensional matri-es X ∈ Rn×vm and Y ∈ Rn×vq with n = sb. Alternatively, the batchrocess data can also be unfolded into two-dimensional matrices′ ∈ Rb×vms and Y′ ∈ Rb×vqs [21]. For on-line monitoring, however,

he alternate unfolding strategy requires that test data should beompleted until the end of each batch and hence variable trajec-ory needs to be estimated. In addition, this method may sufferrom computational difficulty because the number of batches, b,s typically smaller than the product of the number of samplesnd variables vms. Therefore, the unfolded matrices X ∈ Rn×vm and∈ Rn×vq are employed for KPCA, KICA, KPLS and the proposedonitoring method in this study.

.1. Multiway kernel PCA

X can be mapped into high-dimensional feature spaceF throughnonlinear function � ∈ Rm as follows

: xj ∈ Rvm →�(xj) ∈ F (1)

here xj is the jth row vector of X. Then, the covariance matrix ofhe mapped data can be expressed as follows

= 1n

n∑j

�(xj)T �(xj) (2)

here �(xj) is assumed to be mean-centered. Thus the principalomponents in the kernel feature space can be computed by findingigenvectors of C. Instead of solving eigenvalue problem directly,he following eigenvalue decomposition is applied

�i = �in�i (3)

here �i = [˛i1 ˛i2. . .˛in]T ∈ Rn is the orthogonal eigenvectororresponding to the ith largest positive eigenvalues �i and K ∈n×n is the kernel gram matrix defined as

=�(X)�(X)T (4)

ith �(X) = [�(x1)T �(x2)T . . .�(xn)T ]T ∈ Rn×m. Given a new

ample vector xnew, its corresponding score vector is computed asollows

(PCA)new = knewA(PCA) (5)

here A(PCA) = [˛1 ˛2. . .˛a] ∈ Rn×a and knew =�(xnew)�(X)T ∈a denotes the normalized kernel vector for the new sample. The

ollowing MKPCA based T2 and SPE statistics are used as the meas-res of variations in the latent variable and residual subspace forrocess monitoring purpose

2(PCA) = t(PCA)

new

{1

n− 1TT

(PCA)T(PCA)

}−1{t(PCA)

new }T

(6)

PE(PCA) = 1− 2knewA(PCA){t(PCA)new }

T + t(PCA)new AT

(PCA)KA(PCA){t(PCA)new }

T

(7)

here T(PCA) = [t(PCA)1 t(PCA)

2 . . .t(PCA)a ] ∈ Rn×a and the confidence

imits of T2(PCA) and SPE(PCA) can be estimated from F and �2 dis-

ribution, respectively.

Control 24 (2014) 57–71 59

2.2. Multiway kernel ICA

MKPCA is based on the second-order statistic of varianceamong the measurement variables. However, it does not take intoaccount the higher-order statistics and hence the non-Gaussianprocess features may not be efficiently extracted. In order to dealwith non-Gaussian processes, MKICA is developed for nonlinearlyprojecting multivariate process data into latent subspace of statis-tically independent components. KICA algorithm needs whiteningpreprocessing by means of using KPCA [37]. The whitening data inthe feature space F can be obtained as follows

t(ICA) =√

nknewA(ICA)�−1 (8)

where � = diag(�1, �2, . . ., �a) ∈ Ra×a. T(ICA) =[t(ICA)

1 t(ICA)2 . . .t(ICA)

a ] ∈ Rn×a is assumed to be generated aslinear combinations of d(ICA) unknown independent componentsas follows

T(ICA) = SAT(ICA) (9)

with A(ICA) ∈ Rd(ICA)×a denoting the unknown mixing matrix and S =[s1, s2, . . ., sd(ICA)

] ∈ Rn×d(ICA) representing the independent compo-nent matrix. The purpose of KICA is to find a demixing matrixW = [w1, w2, . . ., wd(ICA)

] ∈ Ra×d(ICA)as follows

S = T(ICA)W (10)

where the independent components S have the maximized statis-tical independency in terms of negentropy among each other [16].Given a new sample vector xnew, its corresponding independentcomponent is expressed as

s(ICA)new =

√nknewA�−1W (11)

Further, the I2 and SPE statistics can be defined for process moni-toring as follows

I2(ICA) = s(ICA)

new {s(ICA)new }

T(12)

SPE(ICA) = knewA�−1(I−WA(ICA)) (13)

where the confidence limits for above two statistics can be esti-mated through kernel density estimation strategy [5].

2.3. Multiway kernel PLS

In MKICA method, independent components are calculated byusing high-order statistics such as negentropy among differentmeasurement variables. However, MKICA based process moni-toring method may not be well suited because only the processmeasurement variables are incorporated in the MKICA model whilethe product quality variables are excluded. In order to take intoaccount the nonlinear impact on output quality variables, MKPLStechnique is developed. X can be mapped into high-dimensionalfeature space F through a nonlinear function � ∈ Rm as follows

� : xj ∈ Rvm →�(xj) ∈ F (14)

where xj is the jth row vector of X. The goal of KPLS is to constructthe PLS model in kernel feature space as follows

�(X) = TPT + E (15)

Y = TQT + F (16)

where �(X) = [�(x1)T , �(x2)T , . . ., �(xn)T ]T ∈ Rn×m, T =

[t1, t2, . . ., td] ∈ Rn×d is the score matrix, P = [p1, p2, . . ., pd] ∈Rm×d is the loading matrix of �(X), Q = [q1, q2, . . ., qd] ∈ Rvq×d

is the loading matrix of Y, E ∈ Rn×m is the residual matrix of�(X), and F ∈ Rn×vq is the residual matrix of Y. The nonlinear

6 rocess

ito

m

s

wswc

�

Y

ft

K

wfYt

t

w

saf

T

S

w�

3b

toctpctltiiht

fnt(bLqsi

0 J. Mori, J. Yu / Journal of P

terative partial least squares (NIPALS) algorithm can be performedo determine the ith score vector ti and ui with the followingptimization problem

ax [cov(ti, ui)] = [cov(�(Xi)wi, Yiqi)] (17)

.t.∣∣∣∣ti

∣∣∣∣ = 1,∣∣∣∣ui

∣∣∣∣ = 1 (18)

here cov(ti, ui) represents the covariance matrix between thecore vectors ti ∈ Rn and ui ∈ Rn, and wi ∈ Rm and qi ∈ Rvm denoteeighting vectors. �(Xi) and Yi are deflated matrices that can be

alculated as follows

(Xi+1) =�(Xi)− titTi �(Xi) (19)

i+1 = Yi − titTi Yi (20)

The calculation of the inner products in the high-dimensionaleature space can be avoided by a kernel gram matrix Ki ∈ Rn×n

hat is defined as follows

i =�(Xi)�(Xi)T (21)

here the commonly used kernel function is the Gaussian kernelunction. After the extraction of the score vectors, the matrix Ki andi are deflated in each iteration. Given a new sample vector xnew,he corresponding new score vector is computed as follows

new = knewA(PLS) (22)

here knew =�(xnew)�(X)T ∈ Rn is a kernel vector for the new

ample, A(PLS) = U(TT KU)−1

with U = [u1, u2, . . ., ud] ∈ Rn×d beingscore matrix of Y. The T2 and SPE statistics can then be defined as

ollows

2(PLS) = tnew

{1

n− 1TT T}−1

tTnew (23)

PE(PLS) = 1− 2knewA(PLS)tTnew + tnewAT

(PLS)KA(PLS)tTnew (24)

here the confidence limits of T2 and SPE can be estimated from a2 distribution approximately.

. Quality relevant nonlinear batch process monitoringased on MKQNGLSP approach

Though MKPLS method can capture process nonlinearity fromhe measurement and quality variables, it relies on the second-rder statistic of covariance only so that it may not effectivelyapture non-Gaussian process features. In this study, a novel mul-iway kernel based quality relevant non-Gaussian latent subspacerojection method is developed to overcome the challenges ofonventional MKPLS approach for nonlinear batch process moni-oring. The basic idea of the presented method is to search for theatent directions in high-dimensional kernel feature spaces so thathe statistical dependency in terms of multidimensional mutualnformation between measurement and quality variables is max-mized. Compared to MKPLS, MKQNGLSP takes into account theigher-order statistic of mutual information and thus can effec-ively extract the non-Gaussian features from batch process data.

Since batch processes often have unequal durations across dif-erent batches, the synchronization of batch trajectories may beeeded to solve the issue of batch-to-batch variations. In ordero conduct batch trajectory alignment, the dynamic time warpingDTW) technique can be utilized [19]. Then the three-dimensionalatch process data can be unfolded and scaled, as shown in Fig. 1.

et X ∈ Rb×vm×s and Y ∈ Rb×vq×s be the process measurement anduality data matrices with b batches, s sampling instants, vm mea-urement variables, and vq quality variables. Batch-wise unfoldings first performed to transform matrices X and Y into X ∈ Rb×svm

Control 24 (2014) 57–71

and Y ∈ Rb×svq . Then each column vectors of X and Y are normal-ized to zero-mean and unit-variance so that the mean trajectoryof each batch can be eliminated. Further, the scaled matrices areconverted into new matrices X ∈ Rn×vm and Y ∈ Rn×vq with n = sbthrough variable-wise unfolding, where different sampling instantsare stacked with various batches as row vectors.

After the above unfolded and scaled matrices are obtained, thenext step is to find latent directions in high-dimensional kernelfeature spaces so that the maximized mutual information betweenmeasurement and quality variables is obtained. In order to reducethe process nonlinearity, the measurement and quality variablesare nonlinearly mapped into high-dimensional feature spaces.First, the measurement variables X can be mapped into high-dimensional feature space F through nonlinear mapping function� ∈ Rm as

� : xj ∈ Rvm →�(xj) ∈ F (25)

where xj is the jth row vector of X. The covariance matrix C in thekernel feature space can be expressed as

C = 1n

n∑j

�(xj)T �(xj) (26)

where �(xj) is assumed to be mean-centered. Then principal com-ponents in the kernel feature space can be computed by solving thefollowing eigenvalue problem [5]

Cvi = �ivi (27)

where v1, v2, . . ., vm are the eigenvectors of C ∈ Rm×m correspond-ing to the eigenvalues �1 ≥�2 ≥ . . .≥�m. The eigenvalue �i satisfies

1n

n∑j

�(xj)T {�(xj)vi} = �ivi (28)

while the eigenvector vi can be expressed as linear combination of�(xj) as follows

vi =n∑j

˛ij�(xj)T (29)

where ˛ij is the linear coefficient. From Eqs. (28) and (29), �i =[˛i1, . . ., ˛in]T ∈ Rn can be computed by solving the followingeigenvalue decomposition

K�i = �in�i (30)

where K is kernel gram matrix defined as

[K]ij =�(xi)�T (xj) = K(xi, xj) (31)

and

K =�(X)�(X)T (32)

where K(xi, xj) is a kernel function and �(X) =[�(x1)T , �(x2)T , . . ., �(xn)T ]

T ∈ Rn×m. A commonly used kernelfunction is the Gaussian radial basis function defined as

K(xi, xj) = exp

(−∣∣∣∣xi − xj

∣∣∣∣2c

)(33)

where c is the Gaussian kernel width and its value can be deter-mined by cross-validation strategy. The use of kernel function canavoid the “dimensionality curse” due to the computations of all

pairs of inner products of nonlinear mapping functions. In orderto satisfy the assumption of mean-centered �(xj), mean centeringneeds be conducted on the kernel gram matrix K as follows [5]

K = K− 1nnK− K1nn + 1nnK1nn (34)

J. Mori, J. Yu / Journal of Process Control 24 (2014) 57–71 61

Sampling instants, s

Batches, b

Measurementvariables, v m

b

1 vm 2vmsvm

Batch-wise unfoldingand normalization

Variable-wise unfolding

b

1

2b

vm

...

. . .

proce

w

1

Wr

K

AZ

Z

wp

ww

A

Z

St

[q1, q2, . . ., qd] ∈ Re×d is the loading matrix of V, E ∈ Rn×a isthe residual matrix of Z, F ∈ Rn×e is the residual matrix of V, andd is a number of latent directions. Then the objective is to findweighting vectors w and c from Z and V so that the mutual infor-mation between the score vectors corresponding to measurementand quality variables is maximized as follows

max I(si; ui) = I(Ziwi; Vici) (43)

sb

Fig. 1. Unfoling of three-dimensional batch

here

nn = 1n

⎡⎢⎣

1 · · · 1...

. . ....

1 · · · 1

⎤⎥⎦ ∈ Rn×n (35)

ith the mean-centered kernel gram matrix K, Eq. (3) can beewritten as

�i = n�i�i (36)

fter the eigenvalue decomposition is completed, the score matrix= [z1, . . ., zn] ∈ Rn×n can be obtained as

T = AT K (37)

here A = [˛1, . . ., ˛n] ∈ Rn×n. In this work, the number of princi-al components is determined according to the following criteria

�i∑ni=1�i

≥� (38)

here � represents a user-specified threshold and set to 0.95 in thisork. Thus, A and Z can be extracted as

˜ = [�1, �2, . . ., �a] ∈ Rn×a (39)

= [z1, z2, . . ., za] ∈ Rn×a (40)

imilarly, principal components V ∈ Rn×e for quality variables inhe kernel feature space Y can be computed.

ss data into two-dimensional data matrix.

With the estimated kernel principal components for bothmeasurement and quality variables in high-dimensional featurespaces, Z ∈ Rn×a and V ∈ Rn×e can be further projected onto lower-dimensional latent subspaces as follows

Z = SPT + E (41)

V = SQT + F (42)

where S = [s1, s2, . . ., sd] ∈ Rn×d is the score matrix,P = [p1, p2, . . ., pd] ∈ Ra×d is the loading matrix of Z, Q =

s.t.∣∣∣∣wi

∣∣∣∣ = 1,∣∣∣∣ci

∣∣∣∣ = 1 (44)

6 rocess Control 24 (2014) 57–71

wo

I

wep

caphetnlspiii⎡⎣Fmp

Onev

T

T

w

Ac

l

w

c

˜

˜

2 J. Mori, J. Yu / Journal of P

here I(si, ui) denotes the mutual information between the ith pairf score vectors si and ui. The mutual information is defined as

(si; ui) = H(ui)−H(ui|si) = −∫

ui

f (u)logf (u)ds

+∫

ui

∫si

f (u, s)logf (u|s)f (u, s)

duds (45)

here H(ui) is the marginal entropy of ui, H(ui|si) is the conditionalntropy of ui given si, and f(·) represents a marginal or conditionalrobability density function.

As Eq. (45) contains the complex integrals that are difficult toalculate analytically, a numerical optimization method termeds Nelder-Mead algorithm is adopted to solve the optimizationroblem given in Eq. (43) [29]. However, this algorithm cannotandle the constraints and thus wi and ci are normalized in theach iteration to guarantee the constraint conditions. Furthermore,he mutual information based optimization problem has strongon-linearity and multi-peak feature, which may easily lead the

ocal optimum. To overcome this issue, the multi-start optimizationtrategy is utilized. First, lmax simplices each of which is a specialolytope with a + e + 1 vertices are formed through random initial-

zation. It should be noted that the number of initial points lmax

s a user-specified parameter and is set to 100 in this work. Thentegrated matrix of all weighting vectors is formulated as

w(1)(l)i

w(2)(l)i

. . . w(v)(l)i

. . . w(a+e+1)(l)i

c(1)(l)i

c(2)(l)i

. . . c(v)(l)i

. . . c(a+e+1)(l)i

⎤⎦ ∈ R(a+e)×(a+e+1) (46)

or all l = 1, 2, . . ., lmax, compute w(l)i

and c(l)i

such that the locallyaximized mutual information can be obtained as many times as

ossible. First, w(v)(l)i

and c(v)(l)i

are normalized as

w(v)(l)i

= w(v)(l)i∣∣∣∣∣∣w(v)(l)i

∣∣∣∣∣∣ ∀vc(v)(l)

i= c(v)(l)

i∣∣∣∣∣∣c(v)(l)i

∣∣∣∣∣∣ ∀v(47)

n the basis of the computed mutual information of each pair oformalized weighting vectors, the optimal step sizes and can bestimated from Nelder–Mead algorithm and then the weightingectors are updated as

w(v)(l)i

← w(v)(l)i+�w

c(v)(l)i

← c(v)(l)i+�c

(48)

he above procedure is iterated until w(v)(l)i

and c(v)(l)i

are converged.

hen lth weighting vectors w(l)i

and c(l)i

are set as

˜ (l)i= w(v)(l)

i, c(l)

i= c(v)(l)

i(49)

fter computing w(l)i

and c(l)i

for all l = 1, 2, . . ., lmax, the best wi and

i are chosen as follows

opt = argmaxl

I(Ziw(l)i

; Vic(l)i

) (50)

˜ i = wlopti (51)

i = clopti (52)

Fig. 2. Illustration of the proposed MKQNGLSP approach.

With the obtained ith weighting vectors wi and ci, the scorevectors si and ui can be computed as

si = Ziwi (53)

ui = Vici (54)

Further, the ith loading vectors pi and qi are estimated as

pi =Z

Ti si

sTi si

(55)

qi =V

Ti si

sTi si

(56)

Thus, the matrices Zi and Vi are deflated as follows

Zi+1 = Zi − sipTi and Vi+1 = Vi − siq

Ti (57)

Since si cannot be calculated from Z directly by utilizing wi, thedecomposition matrix R = [r1, r2, . . ., rd] is then defined as

r1 = w1 (58)

ri =i−1∏j=1

(Im − wjpTj )wi i≥2 (59)

where Im is m×m identity matrix. Therefore, the score matrix S canbe computed from the kernel principal components Z as follows

S = ZR (60)

After the MKQNGLSP model is built, it is important to select thenumber of leading latent variables for process monitoring. In thisstudy, the column vectors of and S are sorted according to the nor-

J. Mori, J. Yu / Journal of Process Control 24 (2014) 57–71 63

he pro

mv

I

Adlsr

P = [p1, p2, . . ., pd] ∈ Ra×d (63)

Fig. 3. Schematic diagram of t

alized mutual information between the corresponding columnectors si and ui, which is defined as follows

(si; ui) =I(si; ui)√

H(si)√

H(ui)(61)

fter rearranging the column vectors of S, the number of latent

irections can then be determined. Too few latent directions may

ead to inadequate non-Gaussian features captured in latent sub-pace for fault detection, while too many latent variables can causeedundant information and reduced sensitivity to process faults.

posed MKQNGLSP approach.

Therefore, the best number of latent directions d is chosen to satisfythe following criteria based on the normalized mutual information∑d

i=1I(si; ui)∑ai=1I(si; ui)

≥� (62)

where � is the user-specified threshold and set to 0.95 in this work.Thus, and R can be extracted as

Q = [q1, q2, . . ., qd] ∈ Re×d (64)

R = [r1, r2, . . ., rd] ∈ Ra×d (65)

64 J. Mori, J. Yu / Journal of Process Control 24 (2014) 57–71

ed-ba

T

S

TwIkTomcms

TM

TT

Fig. 4. Process flow diagram of the f

he score matrix S is further estimated as

= KAR (66)

he searching strategy for the non-Gaussian latent directionsithin the nonlinear kernel feature space is illustrated in Fig. 2.

t can be seen that both input and output data are first mapped intoernel feature spaces and then the latent directions are extracted.he aim of searching for latent variables is to maximize the amountf information of u1 in terms of the marginal entropy H(u1) whileinimizing the amount of ambiguity of u1 given s1 in terms of the

onditional entropy H(u1|s1). Such strategy is equivalent to maxi-ize mutual information I(s1; u1) between the score vectors u1 and

1 in the high-dimensional kernel feature space.

able 1easurement and quality variables of the fed-batch penicillin fermentation process.

Variable no. Variable description Variable type

1 Dissolved aeration rate Process variable2 Agitator power Process variable3 Substrate feed temperature Process variable4 Substrate concentration Quality variable5 Dissolved oxygen concentration Quality variable6 Biomass concentration Quality variable7 Penicillin concentration Quality variable8 Culture volume Process variable9 Carbon dioxide concentration Process variable

10 pH Process variable11 Fermenter temperature Process variable12 Generated heat Process variable13 Acid flow rate Process variable14 Base flow rate Process variable15 Cooling water flow rate Process variable

able 2hree test cases of the fed-batch penicillin fermentation process.

Case no. Case description

1 Substrate feed rate is increased by 2.5% from the 200thhour to the end of batch operation

2 Agitator power is increased by 3% from the 250th hour tothe end of batch operation

3 Substrate feed rate is linearly increased with a slope of0.002 from the 60th hour to the end of batch operation

tch penicillin fermentation process.

Given a new sample vector xnew, its corresponding score vectoris computed as follows

snew = knewAR (67)

where knew ∈ Rn denotes the normalized kernel vector for the newsample and is computed below

knew = knew − 11nK− K1nn + 11nK1nn (68)

with

knew =�(xnew)�T (X) (69)

11n =1n

[ 1 · · · 1 ] ∈ Rn (70)

where �(xnew) can be obtained by mapping xnew through a nonlin-ear mapping function �. With the MKQNGLSP model, I2 and SPEstatistics can be derived for batch process monitoring and faultdetection. The I2 statistic aims to detect the process abnormalitiesin the systematic part of the MKQNGLSP model while the SPE statis-tic is designed to capture the abnormality in the residual part of themodel. In MKQNGLSP based monitoring method, the following I2

and SPE indices are proposed:

I2 = RTA

Tk

TnewknewAR (71)

and

SPE =∣∣∣∣∣∣�(xnew)− �(xnew)

∣∣∣∣∣∣2 = 1− 2�(xnew)�T(xnew)

+ �(xnew)�T(xnew) = 1− 2knewA

TsT

new + snewPTKPsT

new

(72)

After the above monitoring indices are defined, appropriate con-

fidence limits need to be derived for isolating abnormal operatingregions from normal batch operation. In this work, the confidencelimits corresponding to I2 and SPE indices are obtained through ker-nel density estimator [5]. Assume that a sequence of I2 or SPE index

J. Mori, J. Yu / Journal of Process Control 24 (2014) 57–71 65

0 5 10 15 20 25 300

0.05

0.1

0.15

0.2

0.25

incipa

Per

cent

age

of E

igen

valu

e

ernel

vd

p

wnf

p

Wcc

ai

Number of Kernel Pr

Fig. 5. Plot of the percentages of eigenvalues of k

alues (ˇ1, ˇ2, . . ., ˇn) are generated from an unknown probabilityensity function p(ˇ) as follows

(ˇ) = 1nD

n∑i=1

K

{ˇ − ˇi

D

}(73)

here ˇ denotes I2 or SPE statistic under consideration, D is the ker-el window width and K is a kernel function. The Gaussian kernel

unction based kernel density estimator can be expressed as

(ˇ) = 1n

n∑i=1

1√2�D

exp

{− (ˇ − ˇi)

2

2D2

}(74)

ith the estimated probability density function and the specifiedonfidence level (1−�)×100 %, the corresponding point with theumulative probability 1−� represents the confidence limit value.

The step-by-step procedure of the presented MKQNGLSPpproach is listed below and the corresponding flowchart is shownn Fig. 3.

1) Collect training data from normal batch operation.2) Convert both process measurement and quality data through

batch-wise unfolding.

3) Normalize unfolded data matrices to zero-mean and unit-

variance.4) Convert the scaled data matrices through variable-wise unfol-

ding.

l Components for Measurement Variables

principal components of measurement variables.

5) Conduct eigenvalue decomposition on the scaled kernel grammatrices for both measurement and quality variables.

6) Estimate kernel principal components for both measurementand quality variables by extracting eigenvectors and eigenval-ues.

7) Compute the multidimensional latent directions in the kernel-principal-components subspaces so that the two sets of latentvariables have the maximized multidimensional mutual infor-mation between the measurement and quality variables.

8) Calculate I2 and SPE indices and estimate the confidence limitsof these statistics through kernel density estimation.

9) For new batch process data, conduct unfolding and normaliza-tion by by using same means and variances from the trainingdata.

10) Compute a newly scaled kernel gram vector from the new batchdata.

11) Estimate latent scores corresponding to measurement vari-ables for the new batch data.

12) Calculate the values of I2 and SPE statistics for the new batchdata.

4. Application example

4.1. Fed-batch penicillin fermentation process

In this work, a fed-batch penicillin fermentation process is uti-lized to examine the performance of the proposed MKQNGLSP

66 J. Mori, J. Yu / Journal of Process Control 24 (2014) 57–71

Table 3Comparison of fault detection rates (%) of three test cases in the fed-batch penicillin fermentation process.

Case no. Fault detection rate (%)

MKPCA MKICA MKPLS MKQNGLSP

T2 SPE I2 SPE T2 SPE I2 SPE

1 47.13 54.86 84.04 24.19 4.74 17.71 91.77 91.272 91.69 27.91 28.90 1.99 68.77 65.12 97.67 86.713 82.23 80.62 93.98 73.57 58.59 68.58 96.92 96.92

Table 4Comparison of false alarm rates (%) of three test cases in the fed-batch penicillin fermentation process.

Case no. False alarm rate (%)

MKPCA MKICA MKPLS MKQNGLSP

T2 SPE I2 SPE T2 SPE I2 SPE

1 0.00 0.00 0.00 0.00 0.25 0.00 0.00 0.000.000.00

agTana

2 0.00 0.00 0.003 0.00 0.00 0.00

pproach for nonlinear batch process monitoring [4]. The dia-ram of the penicillin fermentation process is shown in Fig. 4.

he fermentation process starts with small amount of biomassnd substrate that are added to the bioreactor from the begin-ing of the batch operation. After about 40 h, most of the initiallydded substrate is consumed and then the process is switched from

1 2 3 4 50

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Number of Kernel Principal C

Per

cent

age

of E

igen

valu

e

Fig. 6. Plot of the percentages of eigenvalues of kern

0.60 0.40 3.01 3.010.00 0.00 0.00 0.00

batch to fed-batch operating mode. In the second stage, the sub-strate of glucose is being fed into fermenter continuously under

open-loop condition. Meanwhile, in order to maintain the constanttemperature and pH values (25◦C and 5.0), two proportional-integral-derivative (PID) control loops are implemented in thefermenter for manipulating the acid/base and hot/cold water flow

6 7 8 9 10omponents for Quality Variables

el principal components of quality variables.

J. Mori, J. Yu / Journal of Process Control 24 (2014) 57–71 67

1 2 3 4 5 6 7 8 9 10 11 120

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of Latent Variables

Per

cent

age

of N

orm

aliz

ed M

utua

l Inf

orm

atio

n

Fig. 7. Plot of the percentages and cumulative percentages of normalized mutual information of the extracted non-Gaussian latent variables.

rs0Xstippmst2hmpbou

4

pk

atios, respectively. The duration of each batch is 400 h, while theampling time of both measurement and quality variables are set to.5 h. There are total 11 measurement variables as input variablesand 4 quality variables as output variables Y. All the process mea-

urement and quality variables are listed in Table 1. In our study,he MKQNGLSP model is built from the training data set consist-ng of 10 normal batches. Each batch has small variations in therocess variables. In order to compare the effectiveness of the pro-osed MKQNGLSP approach with the MKPCA, MKICA and MKPLSethods, three test cases are designed with different types of faulty

cenarios, as shown in Table 2. In the first test case, the fermenta-ion process begins with normal operating conditions and then a.5% step increase on substrate feed rate occurs from the 200thour until the end of batch operation. For the second case, the nor-al operation is followed by a fault of 3% step increase on agitator

ower, which begins at the 250th hour and remains until the end ofatch duration. In the last test scenario, a drift error with the slopef 0.002 l/h takes place on substrate feed rate from the 60th hourntil the end of batch operation.

.2. Comparison of batch process monitoring result

After the KPCA model is obtained, the first 12 kernel princi-al components corresponding to measurement variables and 6ernel principal components corresponding to quality variables

are selected, as shown in Fig. 5. Then the MKQNGLSP model isbuilt from 12 kernel PCs for measurement variables and 6 ker-nel PCs for quality variables. Typically, the selected number ofkernel principal components is larger than that of original pro-cess variables, because KPCA extracts principal components fromthe high-dimensional kernel feature space. Further, the number oflatent variables in the obtained MKQNGLSP model is determinedby the normalized mutual information criterion. The percentageand the cumulative percentage of mutual information of differentlatent variables of MKQNGLSP model are shown in Fig. 7. It canbe observed that the first 6 latent directions should be selected astheir corresponding cumulative percentage of normalized mutualinformation is over 95%. For MKPCA, total 9 latent variables whosecumulative percentage of eigenvalues is over 95% are selected. Asfor MKICA, total 9 independent components are chosen becausetheir cumulative percentage of L2 norm of the demixing matrix isover 95% [24]. For MKPLS model, total 11 latent variables with thebest monitoring performance of fault detection rate are selected(Fig. 6).

In the first test scenario, the step fault of substrate feed rateoccurs from the 200th hour. The increase of substrate feed rate

leads to the higher concentrations of biomass and penicillin thanthose under normal operation. The process monitoring results ofthe MKPCA, MKICA, MKPLS and proposed MKQNGLSP methods areshown in Fig. 8. Meanwhile, the fault detection and false alarm rates

68 J. Mori, J. Yu / Journal of Process Control 24 (2014) 57–71

0 100 200 300 4000

2

4

6

8

10

Sample

T2 (MK

PC

A)

0 100 200 300 4000

10

20

30

40

Sample

SP

E (M

KP

CA

)

0 100 200 300 4000

1

2

3x 10

5

Sample

I2 (MK

ICA

)

0 100 200 300 4000

1

2

3

4

5

Sample

SP

E (M

KIC

A)

0 100 200 300 4000

10

20

30

40

50

Sample

T2 (MK

PLS

)

0 100 200 300 4000

100

200

300

400

Sample

SP

E (M

KP

LS)

0 100 200 300 4000

0.2

0.4

0.6

0.8

Sample

I2 (MK

QN

GLS

P)

0 100 200 300 4000

20

40

60

80

100

Sample

SP

E (M

KQ

NG

LSP

)

NormalOperation

Step increasein substrate feed

Fig. 8. Monitoring results of MKPCA (the first row), MKICA (the second row), MKPLS (the third row) and MKQNGLSP (the fourth row) methods in the first test case of thefed-batch penicillin fermentation process.

ocdwTrrtrMwihctttF

f different methods are listed in Tables 3 and 4, respectively. Onean readily see that the MKQNGLSP based I2 is able to accuratelyetect faulty operations with the high fault detection rate of 91.77%hile the false alarm rate of 0.00%. In contrast, the MKPCA based

2 index, the MKICA based I2 index and the MKPLS based T2 indexesult in lower fault detection rates of 47.13% , 84.04 % and 4.74%,espectively. Meanwhile, the MKQNGLSP based SPE index yieldshe high fault detection rate of 91.27% along with the false alarmate of 0.00%. In comparison, the fault detection rates of MKPCA,KICA and MKPLS based SPE index are 54.86% , 24.19 % and 17.71%,hich are significantly lower than that of the MKQNGLSP based SPE

ndex. These results indicate that the proposed MKQNGLSP methodas strong capability to extract nonlinear and non-Gaussian pro-ess features from batch process data. Although the MKICA method

akes into consideration nonlinear and non-Gaussian process fea-ures, it does not incorporate quality variables in the model andhus its performance is worse than that of MKQNGLSP method.urthermore, even though the MKPLS method is able to deal with

process nonlinearity and its model also includes both measurementand quality variables, it does not take into account the high-orderstatistics for capturing non-Gaussian process features so that itsmonitoring performance is not as satisfactory as that of the pro-posed MKQNGLSP approach.

In the second test case, the process operation includes 3% stepincrease of agitator power starting from the 250th hour. The processmonitoring results of the MKPCA, MKICA, MKPLS and MKQNGLSPmethods are compared in Fig. 9 as well as Tables 3 and 4. Itcan be observed that the MKQNGLSP based I2 statistic can detectabnormal operating conditions with the high fault detection rateof 97.67% along with the low false alarm rate of 3.01%. Mean-while, the MKPCA based T2 index, the MKICA based I2 index andthe MKPLS based T2 index can detect the faulty operation with

lower fault detection rates of 91.69% , 28.90 % and 68.77%, respec-tively. On the other hand, the MKQNGLSP based SPE index is ableto trigger the alarms of faulty operation with the fault detectionrate of 86.71% and the false alarm rate of 3.01%. In comparison,

J. Mori, J. Yu / Journal of Process Control 24 (2014) 57–71 69

0 100 200 300 4000

2

4

6

8

10

Sample

T2 (MK

PC

A)

0 100 200 300 4000

5

10

15

20

Sample

SP

E (M

KP

CA

)

0 100 200 300 4000

5

10

x 104

Sample

I2 (MK

ICA

)

0 100 200 300 4000

1

2

3

Sample

SP

E (M

KIC

A)

0 100 200 300 4000

20

40

60

80

Sample

T2 (MK

PLS

)

0 100 200 300 4000

200

400

600

Sample

SP

E (M

KP

LS)

0 100 200 300 4000

0.2

0.4

0.6

Sample

I2 (MK

QN

GLS

P)

0 100 200 300 4000

20

40

60

Sample

SP

E (M

KQ

NG

LSP

)

Step increasein agitator power

NormalOperation

Fig. 9. Monitoring results of MKPCA (the first row), MKICA (the second row), MKPLS (the third row) and MKQNGLSP (the fourth row) methods in the second test case of thefed-batch penicillin fermentation process.

tplmitsaibitpt

amn

he MKPLS based SPE index can only detect 65.12% of faulty sam-les. For MKPCA and MKICA, their SPE indices result in much

ower fault detection rates of 27.91% and 1.99%, respectively. Theain reason why the MKPCA and MKICA show poor performance

s that only the process measurement variables are involved inheir models without any output quality variables. Moreover, theuperiority of the proposed MKQNGLSP method over the MKPLSpproach is due to the fact that the higher-order statistic of mutualnformation rather than the second-order statistic of covarianceetween the process measurement and product quality variables

s maximized while searching for the latent directions. In this way,he non-Gaussian process features can be better extracted by theroposed MKQNGLSP method for quality relevant process moni-oring.

For the third test case, a drift error with the slope of 0.002 l/h isdded to the substrate feed rate starting from the 60th hour, whichakes biomass and penicillin concentrations grow faster than the

ormal trajectories. The monitoring results of the MKPCA, MKICA,

MKPLS and the proposed MKQNGLSP approaches are shown inFig. 10, while the quantitative comparison is given in Tables 3 and4. One can easily see that the MKPCA based T2, MKICA based I2

and MKPLS based T2 indices lead to the fault detection rates of82.23% , 93.98 % and 58.59%, respectively, which are lower thanthat of the MKQNGLSP based T2 index (96.92%). Likewise, thefault detection rates of the MKPCA, MKICA and MKPLS based SPEindices are 80.62% , 73.57 % and 68.58%, respectively. In contrast,the MKQNGLSP based SPE index leads to the fault detection rate of96.92%, which is significant higher than those of the other methods.These results further prove that the proposed MKQNGLSP approachcan effectively extract the non-Gaussian process features in qual-ity relevant latent subspaces and thus has substantially improvedmonitoring capability than the MKPCA, MKICA and MKPLS meth-

ods. All the above three test cases demonstrate that the proposedMKQNGLSP method is able to capture abnormal batch operationwith higher fault sensitivity and detectability than the conventionalMKPCA, MKICA and MKPLS approaches.

70 J. Mori, J. Yu / Journal of Process Control 24 (2014) 57–71

0 100 200 300 4000

20

40

60

Sample

T2 (MK

PC

A)

0 100 200 300 4000

50

100

Sample

SP

E (M

KP

CA

)

0 100 200 300 4000

2

4

6

8

x 105

Sample

I2 (MK

ICA

)

0 100 200 300 4000

5

10

Sample

SP

E (M

KIC

A)

0 100 200 300 4000

100

200

300

400

500

Sample

T2 (MK

PLS

)

0 100 200 300 4000

2000

4000

6000

8000

Sample

SP

E (M

KP

LS)

0 100 200 300 4000

0.5

1

1.5

2

2.5

Sample

I2 (MK

QN

GLS

P)

0 100 200 300 4000

100

200

300

Sample

SP

E (M

KQ

NG

LSP

)

NormalOperation

Linearly increasein substrate feed

F LS (thf

5

scdaedatcvosmsiar

ig. 10. Monitoring results of MKPCA (the first row), MKICA (the second row), MKPed-batch penicillin fermentation process.

. Conclusions

In this paper, a multiway quality relevant non-Gaussian latentubspace projection method is developed for nonlinear batch pro-ess monitoring. The presented approach can identify nonlinearynamics and non-Gaussian relationships between measurementnd quality variables. First, the kernel principal components arextracted from the unfolded and scaled measurement and qualityata sets and thus the nonlinear process dynamics can be char-cterized in the high-dimensional kernel feature space. Secondly,he multidimensional latent directions in the kernel-principal-omponents subspaces corresponding to measurement and qualityariables are searched concurrently by maximizing the higher-rder statistic of mutual information instead of the second-ordertatistic of covariance. Hence, the non-Gaussian features relatingeasurement and quality variables can be well captured. Thirdly, a

et of new monitoring indices are developed to capture abnormal-ties of batch process data within non-Gaussian latent subspacend residual subspace. The proposed MKQNGLSP method incorpo-ates both measurement and quality variables, the latter of which

e third row) and MKQNGLSP (the fourth row) methods in the third test case of the

are not included in the MKPCA and MKICA models. Furthermore,the MKQNGLSP model is characterized with the maximized mutualinformation instead of covariance and thus can effectively capturenon-Gaussian process features that may not be well handled by theconventional MKPLS model.

The presented MKQNGLSP method is applied to a fed-batchPenicillin fermentation process, and the monitoring results demon-strate that the new I2 and SPE indices outperform the MKPCA,MKICA and MKPLS based statistics. The MKQNGLSP approach pro-vides an effective solution for nonlinear and non-Gaussian qualityrelevant batch process monitoring and fault detection. Futureresearch will focus on extending MKQNGLSP method for fault iso-lation and diagnosis of nonlinear batch processes.

References

[1] H. Albazzaz, X.Z. Wang, Statistical process control charts for batch operationsbased on independent component analysis, Ind. Eng. Chem. Res. 43 (2004)6731–6741.

[2] B.R. Bakshi, Multiscale PCA with application to multivariate statistical processmonitoring, AIChE J. 44 (7) (1998) 1596–1610.

rocess

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

[

J. Mori, J. Yu / Journal of P

[3] G. Bastin, D. Dochain, On-line Estimation and Adaptive Control of Bioreactors,Elsevier, Amsterdam, The Netherlands, 1990.

[4] I. Birol, C. Ündey, G. Birol, E. Tatara, A. C inar, A web-based simulator for peni-cillin fermentation, Int. J. Eng. Simul. 2 (2001) 24–30.

[5] C.M. Bishop, Pattern Recognition and Machine Learning, Springer-Verlag, NewYork, NY, USA, 2007.

[6] A. Chen, Z. Song, P. Li, Soft sensor modeling based on DICA-SVR, Lect. NotesComput. Sci. 3644 (2005) 868–877.

[7] J. Chen, K. Liu, On-line batch process monitoring using dynamic PCA anddynamic PLS models, Chem. Eng. Sci. 57 (2002) 63–75.

[8] L.H. Chiang, E.L. Russell, R.D. Braatz, Fault Detection and Diagnosis in IndustrialSystems, in: Advanced Textbooks in Control and Signal Processing, Springer-Verlag, London, Great Britain, 2001.

[9] L.H. Chiang, R. Leardi, R.J. Pell, M.B. Seasholtz, Industrial experiences with mul-tivariate statistical analysis of batch process data, Chemom. Intell. Lab. Syst. 81(2006) 109–119.

10] S.W. Choi, J. Morris, I. Lee, Dynamic model-based batch process monitoring,Chem. Eng. Sci. 63 (3) (2008) 622–636.

11] K. Desai, Y. Badhe, S.S. Tambe, B.D. Kulkarni, Soft-sensor development for fed-batch bioreactors using support vector regression, Biochem. Eng. J. 27 (3) (2006)225–239.

12] L. Di, Z. Xiong, X. Yang, Nonlinear process modeling and optimization based onmultiway kernel partial least squares model, in: Proceedings of IEEE, 2008, pp.1645–1651.

13] F.J. Doyle, Nonlinear inferential control for process applications, J. Proc. Cont. 8(1998) 339–353.

14] J. Flores-Cerrillo, J.F. MacGregor, Multivariate monitoring of batch processesusing batch-to-batch information, AIChE J. 50 (2004) 1219–1228.

15] Z. Ge, Z. Song, Process monitoring based on independent componentanalysis–principal component analysis (ICA–PCA) and similarity factors, Ind.Eng. Chem. Res. 46 (2007) 2054–2063.

16] A. Hyv??rinen, J. Karhunen, E. Oja, Independent Component Analysis, JohnWiley and Sons, New York, NY, USA, 2001.

17] H. Kaneko, M. Arakawa, K. Funatsu, Development of a new soft sensor methodusing independent component analysis and partial least squares, AIChE J. 55(2009) 87–98.

18] M. Kano, S. Tanaka, S. Hasebe, I. Hashimoto, H. Ohno, Monitoring independentcomponents for fault detection, AIChE J. 49 (4) (2003) 969–976.

19] A. Kassidas, J.F. MacGregor, P.A. Taylor, Synchronization of batch trajectoriesusing dynamic time warping, AIChE J. 44 (1998) 864–875.

20] W. Ku, R.H. Storer, C. Georgakis, Disturbance detection and isolation by dynamicprincipal component analysis, Chemom. Intell. Lab. Syst. 30 (1995) 179.

21] J.M. Lee, C.K. Yoo, I.B. Lee, Fault detection of batch processes using mul-tiway kernel principal component analysis, Comput. Chem. Eng. 28 (2004)1837–1847.

22] J.M. Lee, C.K. Yoo, I.B. Lee, Nonlinear process monitoring using kernel principalcomponent analysis, Chem. Eng. Sci. 59 (2004) 223–234.

23] J.M. Lee, C.K. Yoo, I.B. Lee, Statistical monitoring of dynamic processes basedon dynamic independent component analysis, Chem. Eng. Sci. 59 (2004)2995–3006.

24] J.M. Lee, C.K. Yoo, I.B. Lee, Statistical process monitoring with independentcomponent analysis, J. Proc. Cont. 14 (2004) 467–485.

25] J.-M. Lee, C. Yoo, S.W. Choi, P.A. Vanrolleghem, I.-B. Lee, Nonlinear process mon-itoring using kernel principal component analysis, Chem. Eng. Sci. 59 (2004)223–234.

26] B. Lennox, G.A. Montague, H.G. Hiden, G. Kornfeld, P.R. Goulding, Process mon-itoring of an industrial fed-batch fermentation, Biotechnol. Bioeng. 74 (2001)125–135.

27] B. Lennox, H.G. Hiden, G.A. Montague, G. Kornfeld, P.R. Goulding, Process mon-itoring of an industrial fed-batch fermentation, Biotechnol. Bioeng. 74 (2001)125–135.

28] X. Liu, U. Kruger, T. Littler, L. Xie, S. Wang, Moving window kernel PCA for adap-tive monitoring of nonlinear processes, Chemom. Intell. Lab. Syst. 96 (2009)132–143.

29] A. Nelder, R. Mead, A simplex method for function minimization, Comput. J. 7(4) (1965) 308–313.

[

[

Control 24 (2014) 57–71 71

30] P. Nomikos, J.F. MacGregor, Multivariate SPC charts for monitoring batch pro-cesses, Technometrics 37 (1) (1995) 41–59.

31] M.J. Piovoso, K.A. Hoo, Multivariate statistics for process control, IEEE Cont.Syst. Mag. 22 (2002) 8–9.

32] M.J. Piovoso, K.A. Kosanovich, Applications of multivariate statistical methodsto process monitoring and controller design, Int. J. Control 59 (1994) 743–765.

33] M.N. Pons, A. Rajab, J.M. Flaus, J.M. Engasser, A. Cheruy, Comparison of esti-mation methods for biotechnological processes, Chem. Eng. Sci. 8 (1988)1909–1914.

34] S.J. Qin, Statistical process monitoring: basics and beyond, J. Chemometrics 17(2003) 480–502.

35] A. Raich, A. C inar, Statistical process monitoring and disturbance diagnosis inmultivariable continuous processes, AIChE J. 42 (1996) 995–1009.

36] M.M. Rashid, J. Yu, Nonlinear and non-Gaussian dynamic batch process mon-itoring using a new multiway kernel independent component analysis andmultidimensional mutual information based dissimilarity approach, Ind. Eng.Chem. Res. 51 (2012) 10910–10920.

37] X. Tian, X. Zhang, X. Deng, S. Chen, Multiway kernel independent componentanalysis based on feature samples for batch process monitoring, J. Neurocom-puting 72 (2009) 1584–1596.

38] C. Ündey, S. Ertunc, A. C inar, Online batch/fed-batch process performance mon-itoring, quality prediction, and variable-contribution analysis for diagnosis, Ind.Eng. Chem. Res. 42 (2003) 4645–4658.

39] V. Venkatasubramanian, R. Rengaswamy, K. Yin, S.N. Kavuri, A review of pro-cess fault detection and diagnosis: Part I: Quantitative model-based methods,Comput. Chem. Eng. 27 (2003) 293–311.

40] V. Venkatasubramanian, R. Rengaswamy, K. Yin, S.N. Kavuri, A review of processfault detection and diagnosis: Part II: Qualitative models and search strategies,Comput. Chem. Eng. 27 (2003) 313–326.

41] V. Venkatasubramanian, R. Rengaswamy, K. Yin, S.N. Kavuri, A review of processfault detection and diagnosis: Part III: Qualitative models and search strategies,Comput. Chem. Eng. 27 (2003) 327–346.

42] B.M. Wise, N.B. Gallagher, The process chemometrics approach to process mon-itoring and fault detection, J. Proc. Cont. 6 (1996) 329–348.

43] Y. Yao, F. Gao, A survey on multistage/multiphase statistical modeling methodsfor batch processes, Annu. Rev. Control 33 (2009) 172–183.

44] J. Yu, Nonlinear bioprocess monitoring based multiway kernel localized Fisherdiscriminant analysis, Ind. Eng. Chem. Res. 50 (2011) 3390–3402.

45] J. Yu, A nonlinear kernel Gaussian mixture model based inferential monitoringapproach for fault detection and diagnosis of chemical processes, Chem. Eng.Sci. 68 (2012) 506–519.

46] J. Yu, Online quality prediction of nonlinear and non-Gaussian chemical pro-cesses with shifting dynamics using finite mixture model based Gaussianprocess regression approach, Chem. Eng. Sci. 82 (2012) 22–30.

47] J. Yu, A particle filter driven dynamic Gaussian mixture model approach forcomplex process monitoring and fault diagnosis, J. Proc. Cont. 22 (2012)778–788.

48] J. Yu, S.J. Qin, Multiway Gaussian Mixture Model Based Multiphase Batch Pro-cess Monitoring, Ind. Eng. Chem. Res. 48 (18) (2009) 8585–8594.

49] J. Yu, S.J. Qin, Variance component analysis based fault diagnosis of multi-layeroverlay lithography processes, IIE Trans. 41 (2009) 764–775.

50] H. Zhang, B. Lennox, Integrated condition monitoring and control of fed-batchfermentation processes, J. Proc. Cont. 14 (2004) 41–50.

51] X. Zhang, W. Yan, H. Shao, Nonlinear multivariate quality estimation and pre-diction based on kernel partial least squares, Ind. Eng. Chem. Res. 47 (2008)1120–1131.

52] Y. Zhang, Fault detection and diagnosis of nonlinear processes using improvedKernel independent component analysis (KICA) and support vector machine(SVM), Ind. Eng. Chem. Res. 47 (2008) 6961–6971.

53] Y. Zhang, S.J. Qin, Fault detection of nonlinear processes using multiway ker-nel independent component analysis, Ind. Eng. Chem. Res. 46 (23) (2007)

7780–7787.

54] S.J. Zhao, J. Zhang, Y.M. Xu, Performance monitoring of processes with multipleoperating modes through multiple PLS models, J. Proc. Cont. 16 (2006) 763–772.

55] Z. Zhu, Z. Song, A. Palazoglu, Process pattern construction and multi-modemonitoring, J. Proc. Cont. 22 (2012) 247–262.