Journal of Multivariate Analysis 124 (2014) 57–69

Contents lists available at ScienceDirect

Journal of Multivariate Analysis

journal homepage: www.elsevier.com/locate/jmva

D-optimal designs for multiresponse linear models with aqualitative factor

Rong-Xian Yue a,b,c,∗, Xin Liu d, Kashinath Chatterjee e

a Department of Mathematics of Shanghai Normal University, Shanghai 200234, Chinab Scientific Computing Key Laboratory of Shanghai Universities, Shanghai 200234, Chinac Division of Scientific Computation of E-Institute of Shanghai Universities, Shanghai 200234, Chinad College of Science, Donghua University, Shanghai, Chinae Department of Statistics, Visva-Bharati University, Santiniketan, India

h i g h l i g h t s

• Determinant of information matrix of a product design is separated into two parts.• The optimal marginal design is uniform on the index set of the qualitative levels.• D-optimal design for a hierarchically ordered system of multiresponses is given.

a r t i c l e i n f o

Article history:Received 1 December 2012Available online 29 October 2013

AMS subject classifications:62H9962K05

Keywords:D-optimal designsMultiresponse linear modelsQualitative factors

a b s t r a c t

Consider a linear regression model with both quantitative and qualitative factors andan k-dimensional response variable y whose components are equicorrelated for eachobservation. The D-optimal design problem is investigated when the levels of thequalitative factor interact with the quantitative factors. It is shown that the determinant ofthe information matrix of a product design can be separated into two parts correspondingto the two marginal designs. It is also shown that for the hierarchically ordered system ofregression models, the D-optimal design does not depend on the covariance matrix of y.

© 2013 Elsevier Inc. All rights reserved.

1. Introduction

In many experimental situations, especially in engineering, pharmaceutical, biomedical, and environmental research, itbecomes necessary to measure more than one response for each setting of control variables. For example, in the courseof calibration of an apparatus in microwave engineering, several precision transmission line sections are connected tothe apparatus. The connection of each section produces a complex number called a reflection coefficient; the reflectioncoefficients lie on a circle with unknown center and radius, but due to various causes the readings are noisy. Anotherexample is a bioassay experiment that measures a response from different doses of the standard and test preparations.

This work was partially supported by NSFC grant (11071168, 11101077), Special Funds for Doctoral Authorities of Education Ministry(20103127110002), Innovation Program of Shanghai Municipal Education Commission (11zz116), E-Institutes of Shanghai Municipal EducationCommission (E03004), Shanghai Leading Academic Discipline Project, the Fundamental Research Funds for the Central Universities.∗ Corresponding author.

E-mail addresses: yue2@shnu.edu.cn (R.-X. Yue), liuxin@dhu.edu.cn (X. Liu), kashinathchatterjee@gmail.com (K. Chatterjee).

0047-259X/$ – see front matter© 2013 Elsevier Inc. All rights reserved.http://dx.doi.org/10.1016/j.jmva.2013.10.011

58 R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69

Chang [1] studied the properties of D-optimal designs for multiresponse models. Khuri and Cornell [5] devoted a chapter oftheir book to multiresponse experiments.

Designing multiresponse experiments is substantially more complicated than in univariate response settings. Thereare some literature on optimal designs with respect to several optimality criteria for multiresponse linear models withquantitative factors. The reader is referred to the recent papers on optimal multiresponse designs by, e.g., [2,4,8,13,14,10,15,9,11] and the papers cited therein. Lee and Huang [7] demonstrated through an example on chemical study how theD-optimal design may help to design an experiment with both quantitative and qualitative factors more efficiently.

The primary objective of the present study is to extend the work of Lee and Huang [7] to the multiresponse cases. In themultiresponse cases, when an experiment includes a qualitative factor, the effects between the quantitative and qualitativefactors should be taken into consideration. The paper is organized as follows. Section 2 provides notations and preliminaries.Moreover, an illustrative example is presented in this section. Themain result is presented in Section 3. Section 4 deals withD-optimal designs for the first-order models with two responses. To illustrate the application of the main result, Section 5considers D-optimal designs for hierarchically ordered system of regression models. Finally Section 6 provides concludingremarks.

2. Notations and preliminaries

In this paper, the multiresponse linear models include both qualitative and quantitative factors as explanatory variables.In such models, the influence of a qualitative variable is taken into account by allowing some or all of the coefficients of thequantitative variables to differ in accordancewith the specified categories of the qualitative variable. Following [7], a generalmodel of the mean response of the i-th response variable at the j-th level of a s-level qualitative factor with m quantitativefactors described below is more appropriate to be looked at,

E[yi(j, x)] = f Ti1(x)βij + f Ti2(x)γ i, x ∈ X ⊂ Rm, i = 1, . . . , k, j = 1, . . . , s, (1)

where fi1 and fi2 are pi1- and pi2-dimension vectors containing the quantitative effects, respectively, such that fi1 is the partof the regression functions having interactions with the qualitative factor and fi2 is the part invariant at each qualitativelevel. It is to be remarked that the saturated model where fi2 is zero can be taken into account as well, that is, E[yi(j, x)] =

f Ti1(x)βij, i = 1, . . . , k, j = 1, . . . , s. However, there is no difference between fitting such a model and fitting the model forquantitative factors at individual level of the qualitative factor. To illustratemodel (1), let us consider the following example.

Example 1. Suppose there is an experimental situation with bivariate responses. Also suppose that there are two quantita-tive factors and one qualitative factor with s levels. Consider the following linear model

E[y1(j, x)] = β10j + β11jx1 + β12jx2,E[y2(j, x)] = β20j + γ21x1 + γ22x2.

The above model describes the situation where the regression of each of the response variables y1 and y2 is linear on thequantitative factors x1, x2. However, there is a difference between the two regressions. In case of y1, the regression coeffi-cients and the intercept depend on the level of the quantitative factor, whereas in case of y2 only the intercept, and not theregression coefficients, depends on the level of the qualitative factor. Thus the general model (1) is very flexible: (a) it allowsdifferent kinds of regressions for the different components of the response vector, and (b) each component of the responsevector allows some or all of the regression parameters to be dependent on the level of the qualitative factor.

More generally, model (1) can be expressed as

E

y1(j, x)y2(j, x)...yk(j, x)

=

gT1 (j, x) 0 · · · 00 gT

2 (j, x) · · · 0...

......

...

0 0 · · · gTk (j, x)

θ1θ2...θk

,or equivalently

E[y(j, x)] = F(j, x)θ, (2)

where

gTi (j, x) = (eTj ⊗ f Ti1(x), f

Ti2(x)), θi = (βT

i1,βTi2, . . . ,β

Tis, γ

Ti )

T , i = 1, 2, . . . , k,

and ej ∈ Rs is the unit vector whose j-th element is equal to 1 and all others are zeros. Here we assume that the covariancematrix of y(j, x) is Σ . Let Xs = 1, 2, . . . , s be the index set of the qualitative levels, and Ω be the product set Xs × X.Suppose the arbitrary design measure τ onΩ can be expressed as

τ(j, x) = η(j)ξj(x),

R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69 59

where η and ξj are the marginal and the conditional design measures on Xs and X, respectively. Then the quality of thedesign measure τ will be measured through the following information matrix

M(τ ) =

sj=1

η(j)

X

F T (j, x)Σ−1F(j, x)dξj(x) =

sj=1

η(j)Mj, (3)

where

Mj =

X

F T (j, x)Σ−1F(j, x)dξj(x).

It is to be noted that if Σ is the k × k identity matrix Ik, the D-optimality results can be obtained along the line of theresults available in the literature. From the literature review, it is clear that the most commonly used dispersion matrix isan equicorrelation matrix of the formΣ = (1 − ρ)Ik + ρJk, where −1/(k − 1) < ρ < 1 and Jk is the k × k matrix with allelements equal to unity. It is easy to note thatΣ−1

= aIk + bJk, where

a =1

1 − ρand b =

−ρ

(1 − ρ)[1 + (k − 1)ρ].

If τ is considered as a product design and denoted by η × ξ , then under this choice ofΣ , we have

Mj = a

X

F T (j, x)F(j, x)dξ(x)+ b

X

F T (j, x)JkF(j, x)dξ(x),

= a

X

F T (j, x)F(j, x)dξ(x)+ b

X

F∗T (j, x)F∗(j, x)dξ(x),

where

F∗(j, x) = 1Tk F(j, x) = (gT

1 (j, x), gT2 (j, x), . . . , g

Tk (j, x))

and 1k is a k × 1 vector of all ones. It is to be noted that

X

F T (j, x)F(j, x)dξ(x) =

M(11)

j O · · · OO M(22)

j · · · O...

......

...

O O · · · M(kk)j

,where, for 1 ≤ i ≤ k,

M(ii)j =

ejeTj ⊗ M(i)

11 (ξ) ej ⊗ M(i)12 (ξ)

eTj ⊗ M(i)21 (ξ) M(i)

22 (ξ)

,

and

M(i)uv (ξ) =

X

fiu(x)f Tiv (x)dξ(x), u, v = 1, 2.

Also

X

F∗T (j, x)F∗(j, x)dξ(x) =

M(11)

j M(12)j · · · M(1k)

j

M(21)j M(22)

j · · · M(2k)j

......

......

M(k1)j M(k2)

j · · · M(kk)j

,where, for 1 ≤ i < l ≤ k,

M(il)j =

ejeTj ⊗ M(il)

11 (ξ) ej ⊗ M(il)12 (ξ)

eTj ⊗ M(il)21 (ξ) M(il)

22 (ξ)

,

and, for 1 ≤ i < l ≤ k,

M(il)uv (ξ) =

X

fiu(x)f Tlv (x)dξ(x), u, v = 1, 2.

It is to be remarked that, for 1 ≤ i < l ≤ k,

M(li)uv (ξ) = (M(il)

uv (ξ))T and M(li)

j = (M(il)j )T .

60 R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69

The information matrix presented in Eq. (3) can then be simplified to

M(τ ) =

(a + b)A(11) bA(12) · · · bA(1k)

bA(21) (a + b)A(22) · · · bA(2k)...

......

...

bA(k1) bA(k2) · · · (a + b)A(kk)

. (4)

Here, for 1 ≤ i ≤ k,

A(ii) =

sj=1

η(j)ejeTj

⊗ M(i)

11 (ξ)

s

j=1

η(j)ej

⊗ M(i)

12 (ξ)s

j=1

η(j)eTj

⊗ M(i)

21 (ξ) M(i)22 (ξ)

,and, for 1 ≤ i < l ≤ k,

A(il) =

sj=1

η(j)ejeTj

⊗ M(il)

11 (ξ)

s

j=1

η(j)ej

⊗ M(il)

12 (ξ)s

j=1

η(j)eTj

⊗ M(il)

21 (ξ) M(il)22 (ξ)

.The next section presents the main result of this paper.

3. Main result

Let us consider the model described in (1). If the design measure τ is considered as the product design, denoted asτ = η× ξ and the dispersion matrix of the response vector is of the formΣ = (1− ρ)Ik + ρJk, then we have the followingtheorem which will be helpful in developing the remaining sections of this paper.

Theorem 1. For any product design τ with the marginal designs η and ξ on Xs and X respectively, we can write the informationmatrix given in (4) as

M(τ ) ≡

D ⊗ H(k)11 η ⊗ H(k)12ηT

⊗ H(k)21 H(k)22

,

where D = Diag(η(1), η(2), . . . , η(s)), η = (η(1), η(2), . . . , η(s))T ,

H(k)11 =

(a + b)M(1)

11 (ξ) bM(12)11 (ξ) · · · bM(1k)

11 (ξ)

bM(21)11 (ξ) (a + b)M(2)

11 (ξ) · · · bM(2k)11 (ξ)

......

......

bM(k1)11 (ξ) bM(k2)

11 (ξ) · · · (a + b)M(k)11 (ξ)

,

H(k)12 =

(a + b)M(1)

12 (ξ) bM(12)12 (ξ) · · · bM(1k)

12 (ξ)

bM(21)12 (ξ) (a + b)M(2)

12 (ξ) · · · bM(2k)12 (ξ)

......

......

bM(k1)12 (ξ) bM(k2)

12 (ξ) · · · (a + b)M(k)12 (ξ)

,

H(k)22 =

(a + b)M(1)

22 (ξ) bM(12)22 (ξ) · · · bM(1k)

22 (ξ)

bM(21)22 (ξ) (a + b)M(2)

22 (ξ) · · · bM(2k)22 (ξ)

......

......

bM(k1)22 (ξ) bM(k2)

22 (ξ) · · · (a + b)M(k)22 (ξ)

,and H(k)21 = (H(k)12 )

T .

Proof. Let us take k = 2. From (4), we get

M(τ ) =

(a + b)A(11) bA(12)

bA(21) (a + b)A(22)

R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69 61

=

D ⊗ (a + b)M(1)

11 η ⊗ (a + b)M(1)12 D ⊗ bM(12)

11 η ⊗ bM(12)12

ηT⊗ (a + b)M(1)

21 (a + b)M(1)22 ηT

⊗ bM(12)21 bM(12)

22D ⊗ bM(21)

11 η ⊗ bM(21)12 D ⊗ (a + b)M(2)

11 η ⊗ (a + b)M(2)12

ηT⊗ bM(21)

21 bM(21)22 ηT

⊗ (a + b)M(2)21 (a + b)M(2)

22

≡

D ⊗ (a + b)M(1)

11 η ⊗ (a + b)M(1)12 D ⊗ bM(12)

11 η ⊗ bM(12)12

D ⊗ bM(21)11 η ⊗ bM(21)

12 D ⊗ (a + b)M(2)11 η ⊗ (a + b)M(2)

12ηT

⊗ (a + b)M(1)21 (a + b)M(1)

22 ηT⊗ bM(12)

21 bM(12)22

ηT⊗ bM(21)

21 bM(21)22 ηT

⊗ (a + b)M(2)21 (a + b)M(2)

22

≡

D ⊗ (a + b)M(1)

11 D ⊗ bM(12)11 η ⊗ (a + b)M(1)

12 η ⊗ bM(12)12

D ⊗ bM(21)11 D ⊗ (a + b)M(2)

11 η ⊗ bM(21)12 η ⊗ (a + b)M(2)

12ηT

⊗ (a + b)M(1)21 ηT

⊗ bM(12)21 (a + b)M(1)

22 bM(12)22

ηT⊗ bM(21)

21 ηT⊗ (a + b)M(2)

21 bM(21)22 (a + b)M(2)

22

≡

D ⊗ H(2)11 η ⊗ H(2)12ηT

⊗ H(2)21 H(2)22

.

HereM stands forM(ξ). The proof of the general case will follow the same line.

Corollary 1. For any product design τ with the marginal designs η and ξ on Xs and X respectively, the determinant of theinformation matrix of τ can be expressed as

|M(τ )| =

sj=1

η(j) k

i=1pi1 H(k)11

(s−1)H(k)11 H(k)12

H(k)21 H(k)22

, (5)

where, for 1 ≤ i ≤ k, pi1 = dim(M(i)11 ). Moreover, it also follows that, for τ to be optimal, all the elements of η must be equal,

i.e., η(1) = η(2) = · · · = η(s) = 1/s.

In particular, consider the special case, i.e., a model where

gT1 (j, x) = gT

2 (j, x) = · · · = gTk (j, x) = gT (j, x).

It is to be remarked that it is quite logical that we should include as many parameters in the model as possible and thisjustifies the choice of thismodel. Under thismodel assumption, the informationmatrix presented in Eq. (4) can be simplifiedto

M(τ ) = (aIk + bJk)⊗ A(τ ) (6)

where

A(τ ) =

sj=1

η(j)ejeTj

⊗ M11(ξ)

s

j=1

η(j)ej

⊗ M12(ξ)

sj=1

η(j)eTj

⊗ M21(ξ) M22(ξ)

,in which

Muv(ξ) =

X

fu(x)f Tv (x)dξ(x), u, v ∈ 1, 2.

Therefore,

|M(τ )| ∝ |A(τ )|k. (7)

It is clear that A(τ ) is free of the unknown value of the parameter ρ of the dispersion matrix and also of k. In the special casewhen the k regression vectors are the same, the study of a multiresponse model is equivalent to the study of a uniresponsemodel. However, this is not the case when there exists at least two regressor vectors being unequal.

From Eq. (7), it is also clear that the maximization of |M(τ )| amounts to maximization of |A(τ )| in this special case. Thishappens irrespective of the values of the unknown quantities a and b. This leads to a substantial simplification of the optimaldesign problem.

The next section considers the application of Theorem 1 to obtain D-optimal multiresponse designs for response surfacemodels with both qualitative and quantitative factors.

62 R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69

4. D-optimal designs for first-order models with two responses

For response surface designs involving qualitative andquantitative factors, the selection of an appropriate designdependson how the qualitative factors interact with quantitative factors in the model and it is very difficult to obtain D-optimaldesigns. Lee andHuang [7] initiated thework of obtaining D-optimal designs. Theorem1 can be used to extend their findingsto the multiresponse setup and is demonstrated through the following example.

Consider the following linear model

E[y1(j, x)] = f T11(x)β1jE[y2(j, x)] = f T21(x)β2j + f T22(x)γ2

x ∈ X ⊂ Rm, j = 1, . . . , s, (8)

where the symbols are as described just aftermodel equation (1). It is to be remarked that (8) is a special case of (1).Moreover,this is an extension of thework of [7] to themultiresponse setup. The following theoremprovides a simplified expression forthe absolute value of the determinant of the informationmatrix under (8) and also gives a detailing of the proof of Corollary 1under the above special case.

Theorem 2. Under the setup considered in model (8), we have

|M(τ )| = |D|(p11+p21)

M(1)(s−1) M(2)

, (9)

where

M(1)=

(a + b)M(1)

11 bM(12)11

bM(21)11 (a + b)M(2)

11

, and

M(2)=

(a + b)M(1)11 bM(12)

11 bM(12)12

bM(21)11 (a + b)M(2)

11 (a + b)M(2)12

bM(21)21 (a + b)M(2)

21 (a + b)M(2)22

p11 = dim(M(1)

11 ), p21 = dim(M(2)11 ).

Proof. In this case, we have

M(τ ) =

(a + b)A(11) bA(12)

bA(21) (a + b)A(22)

,

=

D ⊗ (a + b)M(1)11 D ⊗ bM(12)

11 η ⊗ bM(12)12

D ⊗ bM(21)11 D ⊗ (a + b)M(2)

11 η ⊗ (a + b)M(2)12

ηT⊗ bM(21)

21 ηT⊗ (a + b)M(2)

21 (a + b)M(2)22

,where η = (η(1), η(2), . . . , η(s))T and D = Diag(η). Therefore,

|M(τ )| =

(a + b)D ⊗ M(1)11

(a + b)G1 −b2

a + bG2

,where

G1 =

D ⊗ M(2)

11 η ⊗ M(2)12

ηT⊗ M(2)

21 M(2)22

, and

G2 =

D ⊗ M(21)

11ηT

⊗ M(21)21

(D ⊗ M(1)

11 )−1 D ⊗ M(12)

11 η ⊗ M(12)12

=

D ⊗ M(21)

11 (M(1)11 )

−1M(12)11 η ⊗ M(21)

11 (M(1)11 )

−1M(12)12

ηT⊗ M(21)

21 (M(1)11 )

−1M(12)11 M(21)

21 (M(1)11 )

−1M(12)12

.

This means, we can write

|M(τ )| =

(a + b)D ⊗ M(1)11

G11 G12G21 G22

=

(a + b)D ⊗ M(1)11

|G11|G22 − G21G−1

11 G12 ,

where

G11 = D ⊗

(a + b)M(2)

11 −b2

a + bM(21)

11 (M(1)11 )

−1M(12)11

,

R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69 63

G12 = η ⊗

(a + b)M(2)

12 −b2

a + bM(21)

11 (M(1)11 )

−1M(12)12

,

G21 = GT12, and G22 = (a + b)M(2)

22 −b2

a + bM(21)

21 (M(1)11 )

−1M(12)12 .

After simplification, it appears

|M(τ )| = |D|(p11+p21)

M(1)(s−1) M(2)

.From (9), it immediately follows that, for τ to be optimal, all the elements of ηmust be equal, i.e., η(1) = η(2) = · · · =

η(s) = 1/s.

Example 1 continued. Suppose that

f T11 = (1, x1, x2), f12 = 0, f21 = 1, f T22 = (x1, x2), X = [−1, 1]2,

and let

µ1 =

X

x1dξ, µ2 =

X

x2dξ, µ12 =

X

x1x2dξ, µ11 =

X

x21dξ, µ22 =

X

x22dξ .

Then

M(1)=

(a + b) (a + b)µ1 (a + b)µ2 b(a + b)µ1 (a + b)µ11 (a + b)µ12 bµ1(a + b)µ2 (a + b)µ12 (a + b)µ22 bµ2

b bµ1 bµ2 a + b

,

M(2)=

(a + b) (a + b)µ1 (a + b)µ2 b bµ1 bµ2(a + b)µ1 (a + b)µ11 (a + b)µ12 bµ1 bµ11 bµ12(a + b)µ2 (a + b)µ12 (a + b)µ22 bµ2 bµ12 bµ22

b bµ1 bµ2 a + b (a + b)µ1 (a + b)µ2bµ1 bµ11 bµ12 (a + b)µ1 (a + b)µ11 (a + b)µ12bµ2 bµ12 bµ22 (a + b)µ2 (a + b)µ12 (a + b)µ22

=

a + b bb a + b

⊗

1 µ1 µ2µ1 µ11 µ12µ2 µ12 µ22

.

It follows that

|M(τ )| = |D|(p11+p21)

M(1)(s−1) M(2)

= |D|

(p11+p21)

[(a + b)4 − b2(a + b)2]

1 µ1 µ2µ1 µ11 µ12µ2 µ12 µ22

(s−1)

× [(a + b)2 − b2]3 1 µ1 µ2µ1 µ11 µ12µ2 µ12 µ22

2

= |D|(p11+p21)(a + b)2s−2

[(a + b)2 − b2]s+2

1 µ1 µ2µ1 µ11 µ12µ2 µ12 µ22

(s+1)

= |D|(p11+p21)(a + b)2s−2

[(a + b)2 − b2]s+2|Mf (ξ)|

(s+1), (10)

where

Mf (ξ) =

1 µ1 µ2µ1 µ11 µ12µ2 µ12 µ22

is the information matrix corresponding to the single response situation with f T (x) = (1, x1, x2). Therefore, the D-optimaldesign is the same as the single response situation. In view of (10), this is intuitively expected because maximization of|M(τ )| is equivalent to that of |Mf (ξ)|, and the terms a, b, etc. arising from the covariance matrix do not play any role here.Moreover, it is well known that the D-optimal design with respect toMf (ξ) is

ξ ∗=

(1, 1) (1,−1) (−1, 1) (−1,−1)1/4 1/4 1/4 1/4

. (11)

Consequently, the D-optimal design τ ∗ is η∗× ξ ∗, where η∗ is the uniform design on Xs.

The next section deals with the D-optimal designs for hierarchically ordered system of regression models.

64 R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69

5. D-optimal designs for hierarchically ordered system of regression models

Consider the hierarchically ordered system of regressionmodels with k responses and a qualitative factor having s levels,say HO(Yjw, cw; G(u)w , q(u)w ), where 1 ≤ j ≤ s, 1 ≤ w ≤ r , u = 1, 2. For 1 ≤ j ≤ s, each response is assumed to belong toone of r classes Yjw , |Yjw| = cw , 1 ≤ w ≤ r ,

rw=1 cw = k. Here we assume that, for each level of the qualitative factor, the

response vector y has homoscedastic variance covariance matrix, say Σ . Moreover, we assume that Σ is such that Σ−1 isof the form

Σ−1= aIk + b1k1T

k ,

where a > 0 and b > −a/k.Suppose, for each j, 1 ≤ j ≤ s and for eachw, 1 ≤ w ≤ r−1,Yjw is associated two systemsG(u)w , |G(u)w | = q(u)w , u = 1, 2 of

regression functions such that G(1)w ∩ G(2)w = ∅, G(1)w ⊆ G(1)w+1 and (G(1)w , G(2)w ) ⊂ (G

(1)w+1, G

(2)w+1) and each response belonging

to Yjw can be expressed as a linear combination of all functions from G(1)w and G(2)w . By putting

πw =

wv=1

cv, 1 ≤ w ≤ r, π0 = 0,

the vectors of the regression functions can be assumed to have the form

f Ti1(x) = (h(1)1 (x), . . . , h(1)

q(1)w(x)), h(1)κ ∈ G(1)w , κ = 1, . . . , q(1)w ,

f Ti2(x) = (h(2)1 (x), . . . , h(2)

q(2)w(x)), h(2)κ ∈ G(2)w , κ = 1, . . . , q(2)w ,

πw−1 + 1 ≤ i ≤ πw, w = 1, . . . , r.

This means, similar to (1), we may write, corresponding to the jth level of the qualitative factor,

E[yi(j, x)] = f Ti1(x)βji + f Ti2(x)γ i, x ∈ X, πw−1 + 1 ≤ i ≤ πw, w = 1, . . . , r. (12)

The following example will clarify the notations stated above as well as the model described in (12). Moreover, it is usedlater to illustrate the technical procedure for finding optimal designs.

Example 2. Consider a polynomial regression on X = [0, 1]. Suppose, for x ∈ X, one measures three response variablesy1, y2 and y3, where y1, y2 follow linear regressions while y3 a quadratic regression. However, there is a difference betweenthese regressions. In case of y1 and y2 the regression coefficient and the intercept depend on the levels of the qualitativefactor whereas in case of y3 only the intercept and the linear component of the regression equation depend on the level ofthe qualitative factor. Under these supposition, we have the following model, derived from model (1),

E[y1(j, x)] = β(1)0j + β

(1)1j x,

E[y2(j, x)] = β(2)0j + β

(2)1j x,

E[y3(j, x)] = β(3)0j + β

(3)1j x + γ x2.

(13)

With reference to the symbols presented at the beginning of this section, we have

k = 3, r = 2, c1 = 2, c2 = 1, q11 = q21 = 2, q22 = 1

and

G(1)1 = G

(1)2 = 1, x, G

(2)1 = Φ, G

(2)2 = x2.

As stated earlier, we measure the worth of a design τ by its information matrix. Similar to (3), we have

M(τ ) =

sj=1

η(j)

X

F T (j, x)Σ−1F(j, x)dξ(x) =

sj=1

η(j)Mj, (14)

where

Mj =

X

F T (j, x)Σ−1F(j, x)dξ(x),

F(j, x) =

F (1)(j, x) 0 · · · 0

0 F (2)(j, x) · · · 0...

......

...

0 0 · · · F (r)(j, x)

,

R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69 65

F (w)(j, x) =

g (w)Tπw−1+1(j, x) 0 · · · 0

0 g (w)Tπw−1+2(j, x) · · · 0...

......

...

0 0 · · · g (w)Tπw(j, x)

, 1 ≤ w ≤ r,

g (w)Ti (j, x) = (eTj ⊗ f Ti1(x), fTi2(x)), πw−1 + 1 ≤ i ≤ πw, 1 ≤ w ≤ r,

θ(w) = (θ(w)Tπw−1+1, θ

(w)Tπw−1+2, . . . , θ

(w)Tπw

)T , 1 ≤ w ≤ r,

and

θ(w)Ti = (β

(w)Ti1 ,β

(w)Ti2 , . . . ,β

(w)Tis , γT

i )T , πw−1 + 1 ≤ i ≤ πw.

Define

M(w)iuv =

X

fiu(x)f Tiv (x)dξ(x), u, v = 1, 2, πw−1 + 1 ≤ i ≤ πw, 1 ≤ w ≤ r,

M(w)i1i2uv

=

X

fi1u(x)fTi2v(x)dξ(x), u, v = 1, 2, πw−1 + 1 ≤ i1 = i2 ≤ πw, 1 ≤ w ≤ r

M(w1w2)i1i2uv

=

X

fi1u(x)fTi2v(x)dξ(x), u, v = 1, 2, πw1−1 + 1 ≤ i1 ≤ πw1 ,

πw2−1 + 1 ≤ i2 ≤ πw2 , 1 ≤ w1 = w2 ≤ r.

Define

M(w)uv (i1, i1) = (a + b)M(w)

i1uv, πw−1 + 1 ≤ i1 ≤ πw, 1 ≤ w ≤ r, u, v = 1, 2,

M(w)uv (i1, i2) = bM(w)

i1 i2uv, πw−1 + 1 ≤ i1 = i2 ≤ πw, 1 ≤ w ≤ r, u, v = 1, 2,

M(w1w2)uv (i1, i2) = bM(w1w2)

i1i2uv, πw1−1 + 1 ≤ i1 ≤ πw1 , πw2−1 + 1 ≤ i2 ≤ πw2 ,

1 ≤ w1 = w2 ≤ r, u, v = 1, 2,

M(w)uv =

M(w)

uv (i1, i2)πw−1+1≤i1,i2≤πw

, 1 ≤ w ≤ r,

M(w1w2)uv =

M(w1w2)

uv (i1, i2)πw1−1+1≤i1≤πw1 , πw2−1+1≤i2≤πw2

, 1 ≤ w1 = w2 ≤ r.

Define, for u, v = 1, 2,

H(r)uv =

M(1)

uv M(12)uv · · · M(1r)

uvM(21)

uv M(2)uv · · · M(2r)

uv...

......

...

M(r1)uv M(r2)

uv · · · M(r)uv

. (15)

The following lemmas will be helpful in proving Theorem 3, the main result, of this section.

Lemma 1. For any measures η and ξ , we can write the information matrix presented in (12) as

M(τ ) ≡

D ⊗ H(r)11 η ⊗ H(r)12ηT

⊗ H(r)21 H(r)22

and

|M(τ )| =

sj=1

η(j) rw=1

cwq(1)w H(r)11

(s−1)H(r)11 H(r)12

H(r)21 H(r)22

,where D and η are as defined in Theorem 1.

Proof. The proof of Lemma 1 follows along the line of the proof of Theorem 1.

Lemma 2. Define

h(1)(x) =

h(1)1 (x), . . . , h

(1)

q(1)r(x)T,

Z (11) =

X

h(1)(x)h(1)T(x)dξ(x),

66 R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69

Z (11)ij = [Iq(1)i, 0]Z (11)[Iq(1)j

, 0]T , 1 ≤ i, j ≤ r,

h(x) =

h(1)1 (x), . . . , h

(1)

q(1)r(x), h(2)1 (x), . . . , h

(2)

q(2)r(x)T,

Z =

X

h(x)hT (x)dξ(x),

Zij =

Iq(1)i

0 0 00 0 Iq(2)i

0

Z

Iq(1)j

0

0 00 Iq(2)j0 0

, 1 ≤ i, j ≤ r,

∆jj = aIcj + b1cj1Tcj , j = 1, . . . , r,

∆ij = b1ci1Tcj , i, j = 1, . . . , r, i = j.

Then we have

H(r)11 =

∆11 ⊗ Z (11)11 · · · ∆1r ⊗ Z (11)1r· · · · · · · · ·

∆r1 ⊗ Z (11)r1 · · · ∆rr ⊗ Z (11)rr

= [∆ij ⊗ Z (11)ij ]1≤i,j≤r , (16)

H(r)11 H(r)12H(r)21 H(r)22

=

∆11 ⊗ Z11 · · · ∆1r ⊗ Z1r

· · · · · · · · ·

∆r1 ⊗ Zr1 · · · ∆rr ⊗ Zrr

= [∆ij ⊗ Zij]1≤i,j≤r , (17)

|H(r)11 | = ap(1)r (a + bcr)q

(1)r

r−1j=1

1 +bcj

a + br

i=j+1ci

q(1)j

rj=1

|Z (11)jj |cj , (18)

where p(1)r =r−1

j=1 cjq(1)j + (cr − 1)q(1)r , and

H(r)11 H(r)12H(r)21 H(r)22

= apr (a + bcr)(q(1)r +q(2)r )

r−1j=1

1 +bcj

a + br

i=j+1ci

(q(1)j +q(2)j )

rj=1

|Zjj|cj , (19)

where pr =r−1

j=1 cj(q(1)j + q(2)j )+ (cr − 1)(q(1)j + q(2)j ).

Proof. The proof of (16) is given below. From (15), we get

H(r)11 =

M(1)

11 M(12)11 · · · M(1r)

11M(21)

11 M(2)11 · · · M(2r)

11...

......

...

M(r1)11 M(r2)

11 · · · M(r)11

.Now, for 1 ≤ w ≤ r , we get

M(w)11 =

M(w)

11 (i1, i2)πw−1+1≤i1,i2≤πw

=

(a + b)Z (11)ww bZ (11)ww · · · bZ (11)ww

bZ (11)ww (a + b)Z (11)ww · · · bZ (11)ww

......

......

bZ (11)ww bZ (11)ww · · · (a + b)Z (11)ww

= ∆ww ⊗ Z (11)ww

R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69 67

and, for 1 ≤ w1 = w2 ≤ r ,

M(w1w2)11 =

M(w1w2)

11 (i1, i2)πw1−1+1≤i1≤πw1 ,πw2−1+1≤i2≤πw2

=

(a + b)Z (11)w1w2

bZ (11)w1w2· · · bZ (11)w1w2

bZ (11)w1w2(a + b)Z (11)w1w2

· · · bZ (11)w1w2...

......

...

bZ (11)w1w2bZ (11)w1w2

· · · (a + b)Z (11)w1w2

= ∆w1w2 ⊗ Z (11)w1w2

.

This completes the proof of (16). The proof of Eq. (17) will follow in a similar manner.To prove (18), it is to be noted that the case r = 1 is trivial. Suppose (18) is true for r = l. Consider the case r = l+ 1. Let

us partition the matrix H(l+1)11 as

H(l+1)11 =

B11 B12B21 B22

,

where

B11 = (∆ij ⊗ Z (11)ij )1≤i,j≤l,

B22 = ∆l+1,l+1 ⊗ Z (11)l+1,l+1 = ∆l+1,l+1 ⊗ Z (11), and

B21 = BT12 = (∆l+1,1 ⊗ Z (11)l+1,1, . . . ,∆l+1,l ⊗ Z (11)l+1,l).

It is to be noted that, for 1 ≤ i, j ≤ l

Z (11)i,l+1 = (Iq(1)i, 0)Z (11)(Iq(1)l+1

)T = [Iq(1)i, 0]Z (11), and

Z (11)l+1,j = (Iq(1)l+1)Z (11)(Iq(1)j

, 0)T = Z (11)(Iq(1)j, 0)T .

It then follows that

B11·2 = B11 − B12B−122 B21

= (∆ij ⊗ Zij)1≤i,j≤l − (∆l+1,1 ⊗ Zl+1,1, . . . ,∆l+1,l ⊗ Zl+1,l)T

× (∆l+1,l+1 ⊗ Z (11))−1[∆l+1,1 ⊗ Zl+1,1, . . . ,∆l+1,l ⊗ Zl+1,l]

= (∆ij ⊗ Zij)1≤i,j≤l − ((∆i,l+1∆−1l+1,l+1∆l+1,j)⊗ (Zi,l+1(Z (11))−1Zl+1,j))1≤i,j≤l

= (∆ij ⊗ Zij)1≤i,j≤l − ((∆i,l+1∆−1l+1,l+1∆l+1,j)⊗ Zij)1≤i,j≤l

= ((∆ij −∆i,l+1∆−1l+1,l+1∆l+1,j)⊗ Zij)1≤i,j≤l

= (∆∗

ij ⊗ Zij)1≤i,j≤l,

where, for 1 ≤ j ≤ l,

∆∗

jj = ∆jj −∆j,l+1∆−1l+1,l+1∆l+1,j = a′Icj + b′1cj1

Tcj

and, for 1 ≤ i = j ≤ l,

∆∗

ij = ∆ij −∆i,l+1∆−1l+1,l+1∆l+1,j = b′1ci1

Tcj

with a′= a and b′

= ab/(a + bcl+1).Applying the induction hypothesis on B11·2, we obtain

|B11·2| = |B11 − B12B−122 B21|

= a′pl(a′+ b′cl)q

(1)l

l−1j=1

1 +b′cj

a′ + b′

li=j+1

ci

q(1)j

lj=1

|Z (11)jj |cj ,

68 R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69

where pl =l−1

j=1 cjq(1)j + (cl − 1)q(1)l . Thus

|B11·2| = aplaq(1)l

1 +

bcla + bcl+1

q(1)l l−1j=1

1 +

ba+bcl+1

cj

1 +b

a+bcl+1

li=j+1

ci

q(1)j

lj=1

|Z (11)jj |cj

= ap∗l

lj=1

1 +bcj

a + bl+1

i=j+1ci

q(1)j

lj=1

|Z (11)jj |cj ,

where p∗

l =l

j=1 cjq(1)j . This implies

|H(l+1)11 | = |B22∥B11 − B12B−1

22 B21|

= |(aIcl+1 + b1cl+11Tcl+1)Z (11)l+1,l+1∥B11 − B12B−1

22 B21|

= apl+1(a + bcl+1)q(1)l+1

lj=1

1 +bcj

a + bl+1

i=j+1ci

q(1)j

l+1j=1

|Z (11)jj |cj .

Furthermore, (19) follows immediately from the proof of (18) by replacing Z (11)ij with Zij. This completes the proof ofLemma 1.

Now we have the following theorem that simplifies the absolute value of the determinant of the information matrixgiven in (14) and under model (12) in a meaningful andmanageable form. Moreover, the proof of this theorem follows fromthe proofs of Lemmas 1 and 2.

Theorem 3. For any measures η and ξ , we can write the information matrix presented in (12) as

|M(τ )| = ψ1(η)ψ2(Σ, ξ),

where

ψ1(η) =

sj=1

η(j) rw=1

cwq(1)w, and

ψ2(Σ, ξ) = ψ21(Σ)

rj=1

|Z (11)jj |(s−1)cj

rj=1

|Zjj|cj ,

and ψ21(Σ) depends only on the elements ofΣ . Thus the factor ψ2(Σ, ξ) is the product of two components, one of whichonly depends onΣ but not on ξ , while the other only on ξ but not onΣ . This is evident fromEqs. (18) and (19). Consequently,the D-optimal designs for the hierarchically ordered system of regression models do not depend onΣ .

Next we consider an example of the hierarchically ordered system with polynomial regression functions. This willillustrate how our approach can reduce the present design problem to a much more meaningful form.

Example 2 continued. It is easy to see that

Z (11)11 = Z (11)22 = Z11 =

µi+j

0≤i,j≤1

, Z22 =

µi+j

0≤i,j≤2

,

where µi+j=

Xxi+jdξ(x). This shows

ψ2(ξ) =

2j=1

|Z (11)jj |(s−1)cj

2j=1

|Zjj|cj

=

µi+j0≤i,j≤1

3s−1 µi+j0≤i,j≤2

.

R.-X. Yue et al. / Journal of Multivariate Analysis 124 (2014) 57–69 69

Furthermore,

log(ψ2(ξ)) = λ

2l=0

βl

l + 1log |Ml(ξ)|,

where

λ = 3s, β0 = 0, β1 = 1 −13s, β2 =

13s,

and

Ml(ξ) =

1

0(1, x, . . . , xl)T (1, x, . . . , xl)dξ(x), 0 ≤ l ≤ 2.

Hence, the optimalmarginal design ξ ∗ can be obtained from Theorem 3.1 in [3] with the prior defined by β = (0, 1−13s ,

13s ).

It is uniquely defined by the canonical moments p1 = 1/2, p2 = (9s + 1)/(9s + 3), p3 = 1/2, p4 = 1, which can be shownto be

ξ ∗=

0 1/2 1

1/2 − 1/(9s + 3) 2/(9s + 3) 1/2 − 1/(9s + 3)

. (20)

6. Concluding remarks

In this work we discuss the D-optimal designs for multiresponse polynomial regression models with both quantitativeand qualitative factors. The product design measures τ = η × ξ are considered to simplify the determinant of the Fisherinformationmatrix. Under the assumption of an equicorrelated covariance structure of the response variables y, the optimalmarginal design η is the uniform design on Xs = 1, . . . , s, while the optimal marginal design ξ substantially depends onhow the qualitative factors interact with quantitative factors in the model. An analytical solution for the optimal ξ is givenin Example 1.

In the presentwork,we also extend the results of [6] on thehierarchically ordered system to include both quantitative andqualitative factors. It shows that the D-optimal product design for the hierarchically ordered system of regression modelsdoes not depend on the covariance matrix of y. Example 2 shows that one can determine the optimal marginal design ξ forthe hierarchically ordered system of polynomial regression models from the results of [3].

Wu and Ding [12] considered an example of a machining process for slab caster rolls in a steel plant which involve bothqualitative and quantitative factors for a single response case. Finally, as a concluding remark, it may be remarked that if themachining process is measured in several directions and one can obtain data for such an experiment, then our result can beused to obtain D-optimal designs.

References

[1] S.I. Chang, Some properties of multi-response D-optimal designs, J. Math. Anal. Appl. 184 (1994) 256–262.[2] F.-C. Chang, M.-N.L. Huang, D.K.J. Lin, H.-C. Yang, Optimal designs for dual response polynomial regressionmodels, J. Statist. Plann. Inference 93 (2001)

309–322.[3] H. Dette, A generalization of D- and D1-optimal designs in polynomial regression, Ann. Statist. 18 (1990) 1784–1804.[4] H.-D. Jin, R.-X. Yue, D- and A-optimal designs for mixture experiments with multiresponse models, J. Shanghai Normal Univ. 37 (2008) 124–130.[5] A.I. Khuri, J.A. Cornell, Response Surfaces: Designs and Analysis, 2nd. ed., Marcel Dekker, New York, 1996.[6] O. Krafft, M. Schaefer, D-optimal designs for a multivariate regression model, J. Multivariate Anal. 42 (1992) 130–140.[7] C.-P. Lee, M.-N.L. Huang, D-optimal designs for second-order response surface models with qualitative factors, J. Data Sci. 9 (2011) 139–153.[8] X. Liu, R.-X. Yue, P-optimal robust designs for multiresponse approximately linear regression, Appl. Math. J. Chinese Univ. 23 (2008) 168–174.[9] X. Liu, R.-X. Yue, A note on R-optimal designs for multiresponse models, Metrika 76 (2013) 483–493.

[10] X. Liu, R.-X. Yue, F.J. Hickernell, Optimality criteria for multiresponse linear models based on predictive ellipsoids, Statist. Sinica 21 (2011) 421–432.[11] X. Liu, R.-X. Yue, Lin, K.J. Dennis, Optimal design for prediction inmultiresponse linearmodels based on rectangular confidence region, J. Statist. Plann.

Inference 143 (2013) 1954–1967.[12] C.F.J. Wu, Y. Ding, Construction of response surface designs for qualitative and quantitative factors, J. Statist. Plann. Inference 71 (1998) 331–348.[13] R.-X. Yue,Model-robust design formultiresponse linearmodelwith possible bias. in: Sixth International Conference on Fuzzy Systems and Knowledge

Discovery, FSKD, vol. 2, pp. 572–576, 2009.[14] R.-X. Yue, X. Liu, IrL -optimal designs for a hierarchically ordered system of regression models, Comput. Statist. Data Anal. 54 (2010) 3458–3465.[15] R.-X. Yue, H. Qin, K. Chatterjee, Optimal U-type design for Bayesian nonparametric multiresponse prediction, J. Statist. Plann. Inference 141 (2011)

2472–2479.