Joint Modeling - Likelihood Approach Revisited

Biometrics 62, 10371043December 2006

DOI: 10.1111/j.1541-0420.2006.00570.x

Joint Modeling of Survival and Longitudinal Data:Likelihood Approach Revisited

Fushing Hsieh,1 Yi-Kuan Tseng,2 and Jane-Ling Wang1,

1Department of Statistics, University of California, Davis, California 95616, U.S.A.2Graduate Institute of Statistics, National Central University, Jhongli City, Taoyuan County 32001, Taiwan

email: [email protected]

Summary. The maximum likelihood approach to jointly model the survival time and its longitudinalcovariates has been successful to model both processes in longitudinal studies. Random effects in the lon-gitudinal process are often used to model the survival times through a proportional hazards model, andthis invokes an EM algorithm to search for the maximum likelihood estimates (MLEs). Several intriguingissues are examined here, including the robustness of the MLEs against departure from the normal randomeffects assumption, and difficulties with the profile likelihood approach to provide reliable estimates for thestandard error of the MLEs. We provide insights into the robustness property and suggest to overcome thedifficulty of reliable estimates for the standard errors by using bootstrap procedures. Numerical studies anddata analysis illustrate our points.

Key words: Joint modeling; Missing information principle; Nonparametric maximum likelihood; Posteriordensity; Profile likelihood.

1. IntroductionIt has become increasingly common in survival studies torecord the values of key longitudinal covariates until the oc-currence of survival time (or event time) of a subject. Thisleads to informative missing/dropout of the longitudinal data,which also complicates the survival analysis. Furthermore, thelongitudinal covariates may involve measurement error. Allthese difficulties can be circumvented by including randomeffects in the longitudinal covariates, for example, through alinear mixed effects model, and modeling the longitudinal andsurvival components jointly rather than separately. Such anapproach is termed joint modeling and we refer the readersto the insightful surveys of Tsiatis and Davidian (2004) andYu et al. (2004) and the references therein. Numerical studiessuggest that the joint maximum likelihood (ML) method ofWulfsohn and Tsiatis (1997), hereafter abbreviated as WT, isamong the most satisfactory approaches to combine informa-tion. We focus on this approach in this article and addressseveral intriguing issues.

The approach described in WT is semiparametric in thatno parametric assumptions are imposed on the baseline haz-ard function in the Cox model (Cox, 1972), while the randomeffects in the longitudinal component are assumed to be nor-mally distributed. An attractive feature of this approach, asconfirmed in simulations, is its robustness against departurefrom the normal random effects assumption. It is in fact as ef-ficient as a semiparametric random effects model proposed bySong, Davidian, and Tsiatis (2002). These authors called forfurther investigation of this intriguing robustness feature. Weprovide a theoretical explanation in Section 2 and demon-strate it numerically in Section 4. A second finding of this

article is to point out the theoretical challenges regardingefficiency and the asymptotic distributions of the paramet-ric estimators in the joint modeling framework. No distribu-tional or asymptotic theory is available to date, and even thestandard errors (SE), defined as the standard deviations ofthe parametric estimators, are difficult to obtain. We explorethese issues in Section 3, highlight the risks when adoptinga heuristic proposal in the literature to estimate the SE, andprovide numerical illustrations in Section 4. A bootstrap pro-cedure is proposed instead to estimate the standard errors.Furthermore, in Section 5, a data example demonstrates thediscussed issues as well as the effectiveness of the bootstrapprocedure.

2. Robustness of the Joint Likelihood ApproachWithout loss of generality, we assume a single time-dependentcovariate, Xi (t), for the ith individual with i = 1, . . . , . . . ,n.The survival time Li is subject to usual independent randomcensoring by Ci , and we observe (Vi , i) for the ith individual,where Vi = min(Li , Ci ) and i = 1(Li Ci ). The longitu-dinal processes are scheduled to be measured at discrete timepoints tij for the ith individual, and are terminated at theendpoint Vi . Hence, the schedule that is actually observed isti, = {ti1, . . . , timi} with timi V i < timi+1, and no longi-tudinal measurements are available after time Vi , promptingthe need to incorporate the survival information and thus thejoint modeling approach to combine information. We furtherassume a measurement error model for the longitudinal co-variate, so in reality what is actually observed is

Wij = Xi(tij) + eij = Xij + eij , j = 1, . . . ,mi. (1)

C 2006, The International Biometric Society 1037

1038 Biometrics, December 2006

Here eij is measurement error that is independent of Xij andhas a parametric distribution, such as N(0, 2e). Hereafter, weuse boldface vector symbols Wi = (W i1, . . . ,W imi) to denotethe observed longitudinal data. To recover the unobservedXi (t), we assume a parametric random effects model with ak-dimensional random effects vector bi g, denoted by Xi (t;bi) = Xi (t). The covariate history Xi(t;bi) = {Xi(s;bi) | 0 s t} is then related to the survival time through a time-dependent Cox proportional hazards model. The hazard func-tion for the ith individual is thus

i(t | Xi(t;bi)) = 0(t)eXi(t;bi). (2)The joint modeling of the longitudinal and survival parts isspecified by (1) and (2). The parameter that specifies thejoint model is = (, 0, ,

2e), where the baseline 0 is

nonparametric.Let f(Wi; , e) and f(Wi |bi; 2e) be, respectively, the

marginal and conditional density of Wi, and f(Vi , i |bi, ,0) the conditional p.d.f. (probability density function) corre-sponding to (2). Under the assumptions of an uninformativetime schedule as illuminated in Tsiatis and Davidian (2004),the contribution of the ith individual to the joint likelihood is

Li() = f(Wi;, e)Ei{f(Vi,i |bi; )}, (3)where Ei denotes conditional expectation with respect to theposterior density of bi given Wi, which is g(bi |Wi; , 2e) =f(Wi |bi; 2e)g(bi)/f(Wi; , 2e).

Two facts emerge. First, Ei{f(Vi , i |bi; )} carries infor-mation on the longitudinal data. If it is ignored, the marginalstatistical inference based on f(Wi; , 2e) alone would be in-efficient and even biased (due to informative dropout). Thissheds light on how a joint modeling approach eliminates thebias incurred by a marginal approach and also why it is moreefficient. Second, the random effect structure (g) and Wiare relevant to the information for survival parameters ( and0) only through the posterior density g(bi |Wi, , 2e), whichcan be approximated well by a normal density through aLaplace approximation technique (Tierney and Kadane, 1986)when reasonably large numbers of longitudinal measurementsare available per subject. This explains why WTs proce-dure is robust against departure from the normal prior as-sumption. Similar robust features were observed for otherlikelihood-based procedures (Solomon and Cox, 1992; Breslowand Clayton, 1993; Breslow and Lin, 1995). See Section 4 forfurther discussion.

3. Relation of EM Algorithm and Fisher InformationDirect maximization of the joint likelihood in (3) is impos-sible due to the nonparametric component 0. However, thenonparametric maximum likelihood estimate (MLE) of 0(t),as defined in Kiefer and Wolfowitz (1956), can be used andit has discrete mass at each uncensored event time Vi . Thesemiparametric problem in (3) is thus converted to a para-metric problem with the parameter representing 0 being adiscrete probability measure of dimension in the order of n.The expectation maximization (EM) algorithm was employedsuccessfully in WT to treat the unobserved random effects bias missing data, leading to a satisfactory nonparametric MLEapproach; compare formulae (3.1)(3.4), and the subsequentdiscussion on page 333 of WT. While there are some com-

putational costs incurred by the imputation of the missingrandom effects in the EM algorithm, the real challenge lies intheory. A major challenge is the high-dimensional nature ofthe baseline hazards parameter so that standard asymptoticarguments for MLEs do not apply. A profile likelihood ap-proach would be an alternative but it encounters difficultiesas well, as elaborated below.

3.1 Implicit Profile EstimatesWe begin with the nonparametric MLE denoted as0(, e, ), given the parameter (, e, ). Substituting thisnonparametric MLE into (3) to produce a profile likelihoodL(, e, , 0(, e, )) and then maximizing this likelihoodwould yield profile estimates of (, e, ). However, for theprofile approach to work as in the classic setting, the pro-file likelihood L(, e, , 0(, e, )) with the nonparametricMLE 0 in place should not involve 0. Unfortunately, this isnot the case here because the nonparametric MLE 0(, e, )cannot be solved explicitly under the random effects structure.Instead, the EM algorithm is employed to update the profilelikelihood estimate for 0(t). The resulting estimate is

0(t) =

ni=1

i1(Vi=t)n

j=1

Ej [exp{Xj(t;bj)}]1(Vit). (4)

Strictly speaking, this is not an estimate as it involves func-tions in the E-step, Ej , which are taken with respect to theposterior density involving 0(t) itself. Specifically, the poste-rior density is

h(bi |Vi,i,Wi, ti; )

=g(bi |Wi, ti;, 2e

)f(Vi,i |bi, , 0)

g(bi |Wi, ti;, 2e

)f(Vi,i |bi, , 0) dbi

. (5)

We can now see that (4) yields an implicit profile estimatesince Ej involves 0(t). The implication is that although apoint estimator of 0 can be derived via the EM algorithm (4),the usual profile approach to calculate the Fisher informationcannot be employed. The asymptotic covariance matrix for 0cannot be evaluated through derivatives of the implicit pro-file likelihood as in traditional situations, where an explicitform of profile likelihood exists. We will resume this informa-tion issue in Section 3.2 after we discuss the remaining profileestimates.

Likewise, the maximum profile likelihood estimates of and 2e are both implicit because the posterior expectationsEj involve both parameters. The estimation of is more com-plicated. To see this, note that only the second term, Ei , in(3) involves and Ei involves the posterior density of bi ,given Wi . This posterior density is the g-function in (5) andnot the posterior density h. Thus, Ei and Ei are different, butafter some rearrangement the score equation for can be ex-pressed as S =

ni=1 Ei{Sc(;0,bi)}, where Sc(;0,bi) =n

i=1

log f(Vi,i |bi, , 0).

Joint Modeling of Survival and Longitudinal Data 1039

Substituting 0(t) by 0(t) in equation (4), the maximumprofile score of is again in implicit form and is denoted as

SIP =

ni=1

Ei{Sc(; 0(t),bi)}

=

ni=1

i[Ei{Xi(Vi;bi)}]

nj=1

Ej [Xj(Vi;bi) exp{Xj(Vi;bi)}]1(VjVi)

nj=1

Ej [exp{Xj(Vi)}]1(VjVi). (6)

The EM algorithm then iterates between the E-step, toevaluate the conditional expectations Ei with parameter val-ues obtained from the previous iteration (k1), and theM-step, to update the estimated values via the above scoreequation. Since (6) has no closed-form solution, the NewtonRaphson method is applied: k = k1 + I

1k1

SIPk1

, where

SIPk1

is the value of the incomplete profile score in (6) with

= k1, and the slope I1k

is obtained through the following

working formula:

IW

=

ni=1

Ei

{

Sc(;0,bi)

}

=

ni=1

nj=1

Ej[Xj(Vi;bi)2 exp{Xj(Vi;bi)}

]1(VjVi)

nj=1

Ej [exp{Xj(Vi;bi)}]1(VjVi)

n

j=1

Ej [Xj(Vi;bi) exp{Xj(Vi;bi)}]1(VjVi)

nj=1

Ej [exp{Xj(Vi;bi)}]1(VjVi)

2

.

(7)

The above iterative procedure is implemented until a con-vergence criterion is met. We next examine what has beenachieved by the EM algorithm at this point.

3.2 Fisher InformationThe iterative layout of the EM algorithm is designed toachieve the MLEs and hence the consistency of parameters(, , 2e) and the cumulative baseline hazard function. Thereis no complication on this front. However, the working formulaIW in (7) at the last step of the EM algorithm has been sug-gested in the literature to provide the precision estimate forthe standard deviation (standard error) of the -estimatorvia (Iw )

1/2. This is invalid. It was derived by taking thepartial derivative of the implicit profile score in equation (6)with respect to , by treating the conditional expectation

Ei [] as if it does not involve . There are two gaps. First, Eidoes involve through the posterior density, so the properway to take the derivative of (6) should involve the multi-plication rule. Second, since (6) is an implicit score, its par-tial derivative with respect to does not yield the correctFisher information. Instead, the projection method needs tobe applied to the Hessian matrix, 2

2logL(), based on the

likelihood L() =

n1 Li () to properly pin down the correct

Fisher information.For illustration purposes we consider a simpler case and

assume for the moment that and 2e are known so that

= (, 0). For this denote I = 2

2logL(), I0 =

20

logL(), and define I00 and I0 similarly. The cor-rect projection to reach the Fisher information for is I I0 [I00 ]

1I0 , which involves inverting a high-dimensionalmatrix I00 . In the general situation with four unknown para-metric components the correct Fisher information would beeven more difficult to compute, as this Hessian matrix has4 4 block matrices corresponding to the four parametriccomponents in , and the projection would be complicated.We recommend bootstrapping to estimate the standard de-viation for all finite-dimensional parameters (, e, ) andillustrate its effectiveness in Section 4.

We close this section by further showing that the workingSE, (IW )

1/2, is smaller than the real SE based on the sampleFisher information of . Hence statistical inference based on(Iw )

1/2 would be too optimistic. Note that the contributionof the ith individual to the true Hessian is

Ii(,) =

Ei[S

c(;0,bi)]

= Ei

[

Sc(;0,bi)

]Ei

[Sc(;0,bi)Sh(;0,bi)

], (8)

with

Sh(;0,bi) =

log h(bi |Vi,i,Wi, ti; )

= Sc(;0,bi) Ei{Sc(;0,bi)

}(9)

being the score function pertaining to the posterior densityh(bi |Vi , i, Wi, ti; ) in (5). It is interesting to note thatthe second term in (8) is equal to Vari[S

c(; 0, bi)], the con-ditional variance with respect to the posterior density, whichis also equal to the amount of Fisher information of con-tained in this posterior density. Recall from the first equalityin (7) that IW is the sample Fisher information of , pertain-ing to the likelihood of f(Xi , i | bi , ) in the parametric Coxmodel. It now follows from (7)(9) that the true Hessian is

I = IW

ni=1

Vari{Sc(;0,bi)

}. (10)

The implication of this relation is that under the joint mod-eling with random effects structure on the covariate Xi, someinformation is lost and that this loss is unrecoverable. There-fore, IW is not the true Hessian, and would be too large whencompared with the true Hessian, I , in (10). The actualamount of information loss,

ni=1 Vari{Sc(;0,bi)} in (10),


Table 1Simulated results for case (1) when the longitudinal data have normal random effects

( 1422, 2e) (422,

2e) (22,

2e) (22, 4

2e) (22,

14

2e) (22, 0

2e)

(0.0025, 0.1) (0.04, 0.1) (0.01, 0.1) (0.01, 0.4) (0.01, 0.025) (0.01, 0)

SD for 0.132 0.098 0.112 0.169 0.108 0.103Mean of (IW

)1/2 0.118 0.085 0.104 0.103 0.103 0.102

Mean of 1.008 0.998 1.004 1.007 0.984 0.987Divergence 0% 0% 0% 4% 0% 0%

involves the posterior variances. Intuitively, we would expectthese posterior variances to increase as the measurement er-ror increases, and this is confirmed in the numerical studyreported in Tables 1 and 2 in Section 4.

This missing information phenomenon, first introduced byOrchard and Woodbury (1972), applies to all the finite-dimensional parameters such as and 2e, as well as 0(t)in the setting considered in this article. It is not unique to thejoint modeling setting and would persist even for a fully para-metric model whenever the EM algorithm is employed. Louis(1982) provided a specific way to calculate the observed Fisherinformation, but this would be a formidable task under oursetting due to the large number of parameters involved.

4. Simulation StudyIn this section, we examine the gap between the working SEformula (IW )

1/2 for and a reliable precision estimate fromthe Monte Carlo sample standard deviation. A second goal isto explore the robustness issue in Section 2. Three cases wereconsidered in the simulations. In all cases, a constant baselinehazard function is used for 0 with measurement error eij N(0, 2e), where = 0,

14 , 1, or 4. The longitudinal covariate

Xi (t) = b1i + b2it is linear in t. The random effects bi = (b1i,b2i) were generated from either a bivariate normal distribu-tion (with E(b1i) = 1, E(b2i) = 2, var(b1i) = 11, var(b2i) =22 with =

14 , 1, or 4, and cov(b1i, b2i) = 12), or a trun-

cated version where b2i is restricted to be nonnegative. Thevarious values of and allow us to examine the effects ofthe variance components on the accuracy of the working for-mula. The case of no measurement error ( = 0) illustratesthat the working formula is now correct as the random effectsare determined from the observations Wi so that there is noinformation loss.

In each setting, the sample size is n = 200 and the numberof replications is 100. The censoring time for each subject isgenerated from exponential distribution with mean 25 for thefirst setting and 110 for the other two settings, resulting in

Table 2Simulated results for case (2) with nonsparse longitudinal data and nonnormal random effects

( 1422, 2e) (422,

2e) (22,

2e) (22, 4

2e) (22,

14

2e) (22, 0

2e)

(0.003, 0.6) (0.048, 0.6) (0.012, 0.6) (0.012, 2.4) (0.012, 0.15) (0.012, 0)

SD for 0.130 0.111 0.119 0.196 0.114 0.103Mean of (IW

)1/2 0.113 0.090 0.100 0.098 0.103 0.103

Mean of 0.977 0.981 0.995 0.987 0.992 1.002Divergence 0% 1% 0% 3% 0% 0%

about 25% censoring. The EM algorithm procedure used hereis the same as in WT except for the numerical integrationto evaluate the conditional expectation in the E-step. Insteadof using the GaussHermite quadrature formula we appliedMonte Carlo integration, as suggested by Henderson, Diggle,and Doboson (2000) to calculate conditional expectations. Pa-rameters and time schedules for the longitudinal and survivalparts under the three settings are specified below.

(a) tij = (0, 1, . . . , 12), = 1, 0 = 0.001, 2e = 0.1, (1,

2) = (2, 0.5), and (11, 12, 22) = (0.5, 0.001, 0.01).(b) tij = (0, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80), =

1, 2e = 0.6, 0 = 1, (1, 2) = (4.173, 0.0103), and(11, 12, 22) = (4.96, 0.0456, 0.012).

(c) Same as (b) except that we reduce 2e to 0.15, and foreach subject we only select the measurement at an ini-tial time point and randomly one or two more pointsbetween the initial and event time.

The first setting yields normally distributed random effectswhile the second one yields very skewed truncated-normalrandom effects (46% of the positive b2i were discarded), butwith a sufficient amount of longitudinal measurements to in-voke the robustness feature as discussed in Section 2. Thethird case also yields truncated-normal random effects, butwith at most three longitudinal measurements per subject.This sparse design for the longitudinal measurements con-firms our remark in Section 2 that it has an adverse effect tothe robustness of a likelihood-based procedure. The choice ofa much smaller 2e for the sparse design is due to the highrates of nonconvergence observed for the EM algorithm (over40% when the same 2e as in case [b] was used). A smallernonconvergence rate as reported in Table 3 provides a stablebackground for illustrating the breakdown of the procedureagainst model violations.

The simulation results are summarized in Tables 13 corre-sponding to the three settings described above. We focus onthe parameter to save space. Nonconvergence rates of the


Table 3Simulated results for case (3) with sparse longitudinal data and nonnormal random effects

1 2 11 12 22 2e

Simulated value 1 4.173 0.0103 4.96 0.0456 0.012 0.15Empirical target 1 4.5032 0.0913 4.7950 0.0175 0.0046 0.1533Mean 1.3498 4.4172 0.2366 3.4356 0.0130 0.0067 3.2903SD 0.1647 0.1573 0.0169 0.4413 0.0246 0.0015 0.3574

EM algorithm are in the fourth row of Tables 1 and 2, andmean of the 100 -estimates in the third row. This suggeststhat the EM algorithm works well in these settings with lownonconvergence rates, and the likelihood-based joint model-ing approach indeed provides unbiased estimates under thetrue random effects distribution setting in Table 1. Whenthe normal random effects assumption is violated (as in Ta-ble 2), it provides roughly unbiased estimates, a confirmationof the robustness of the procedure. The same robustness phe-nomenon was observed in Table 1 of Tsiatis and Davidian(2004) where the random effects were a mixture of two bivari-ate normal distributions leading to symmetric but bimodaldistributions. Our setting in (b) has a skewed (truncated-normal) random effects distribution and thus complementstheir findings. Other simulations not reported here with heav-ier tail distributions, such as multivariate t-distributions, en-joy the same robustness property as long as the longitudinalmeasurements are dense enough. Note that there are at most13, and often much fewer, longitudinal measurements for eachindividual in Table 2, so this is a rather encouraging findingin favor of the normal likelihood-based approach.

The SD reported in the first row of Tables 1 and 2 corre-sponds to the sample standard deviation of the -estimatesfrom 100 Monte Carlo samples, and the working SE reportedin the second row is the mean of the 100 estimated (IW

)1/2

in equation (7), with all unknown quantities estimated at thelast step of the EM algorithm. From these two tables, we cansee that the working SE formula underestimates the stan-dard deviation of in all cases except in the last column,corresponding to no measurement error. The size of observeddifferences is directly related to the size of the measurementerrors (see columns 36). The discrepancy ranges from almostnone (when measurement error is small or 0) to as large as40% in Table 1 and 50% in Table 2. This confirms empiricallythat the working slope should not be used for statistical infer-ence of , especially in the case of large 2e . We re-emphasizethat this applies equally to all other parameters among(, e, 0).

An interesting finding from the first three columns ofTables 1 and 2 is that the discrepancies are noticeable but theydo not vary much when one changes only the variance compo-nent of b2i . Larger 22, however, gives more precise estimatesfor as the associated SD is smallest for the second column inboth tables. This is perhaps not surprising and is consistentwith usual experimental design principles in cross-sectionalstudies. Typically, larger variations in covariates lead to moreprecise regression coefficients, which would correspond to in the joint modeling setting. So our simulations suggest thatthe same design principle continues to hold in this setting.

Increased variation among individual longitudinal processesprovides more information on how the longitudinal covariateof a subject relates to its survival time.

The robustness enjoyed in situations with rich longitudinaldata information is in doubt for the sparse data situation incase (c). Table 3 shows 35% bias in estimating for this case.The mean of the working SE is 0.1163 while the simulated SDis 0.1647, and a large discrepancy is exhibited here. In addi-tion to , we were also curious about the robustness of otherparameters. Since the original simulated values (reported inrow 1) are no longer the target because of truncation, the tar-get values are estimated empirically in Table 3 and reportedin row 2. This then should be the comparison base for the biasof other parameters, which reveals significant, and sometimesserious, biases as well.

4.1 Bootstrap Estimate for the Standard ErrorAs there is no reliable SE formula available due to the the-oretical difficulties mentioned in Section 3, we propose abootstrap method to estimate the SE. The bootstrap sample{Y i , i = 1, . . . ,n}, where Yi = (Vi , i, Wi, ti), is gener-ated from the original observed data as defined in Section 2.The EM algorithm is then applied to the bootstrap sample,Y i , i = 1, . . . ,n, to derive the MLE

, and this is repeatedB times. The bootstrapped SD estimates for come fromthe diagonal elements of the covariance matrix, Cov() =

1/(B 1)B

b=1(b b)(b b)T , where b =

Bb=1

b/B.

To check the stability of the above bootstrap procedure weconducted a small simulation study, using the design in thefirst setting as reported in the third column of Table 1. Ofthe 100 Monte Carlo samples, we resampled B = 50 boot-strap samples for each single Monte Carlo sample. The SDfor based on the 100 Monte Carlo samples is 0.1193 and themean of the 50 bootstrap standard deviation estimates for is 0.1210 with a standard deviation of 0.0145. This suggeststhat the bootstrap SD is quite stable and reliable as it is closeto the Monte Carlo SD. Moreover, the EM algorithm has alow nonconvergence rate of 0.44%, which is 22 out of the 5000Monte Carlo bootstrap samples.

5. Egg-Laying Data AnalysisThe lifetime (in days) and complete reproductive history, interms of number of eggs laid daily until death, of 1000 femalemedflies (Mediterranean fruit fly) were recorded by Careyet al. (1998) to explore the relation between reproduction andlongevity. Since we have the complete egg-laying history with-out missing or censored data, we could use standard softwareto check the proportional hazards assumption without relyingon the last-value-carry-forward procedure, known to be prone


Table 4Analysis of lifetime and log-fecundity of medflies with lifetime reproduction less than 300 eggs. The bootstrap

SD and the working SE, (IW

)1/2, are reported in the last two rows.

1 2 11 12 22 2e

EM estimate 0.5597 0.6950 0.0542 0.7656 0.1123 0.0196 0.9883Bootstrap mean 0.5676 0.6932 0.0541 0.7621 0.1121 0.0198 0.9858Bootstrap SD 0.0921 0.0736 0.0156 0.1056 0.0207 0.0043 0.0606(IW

)1/2 0.0801 0.0663 0.0141 0.0988 0.0176 0.0039 0.0495

to biases. The proportional hazards assumption failed for fliesthat are highly productive, so we restrict our attention to theless productive half of the flies. This includes female medfliesthat produced less than a total of 749 eggs during their life-times. To demonstrate the effects of wrongly employing theworking SE formula to estimate the standard errors of the es-timating procedures in Section 3, we further divide these fliesinto two groups. The first group includes 249 female medfliesthat have produced eggs but with a lifetime reproduction ofless than 300 eggs. The second group includes the 247 fe-male medflies producing between 300 and 749 eggs in theirlifetimes.

As daily egg production is subject to random fluctuationsbut the underlying reproductive process can be reasonably as-sumed to be a smooth process, the measurement error model(1) is a good prescription to link the underlying reproductiveprocess to longevity. Because we are dealing with count data,it is common to take the log transformation, so we take W (t)= log{m(t) + 1}, to avoid days when no eggs were laid. An ex-amination of the individual egg-laying trajectories in Careyet al. (1998) suggests the following parametric longitudinalprocess for X(t):

X(t) = b1 log(t) + b2(t 1),

where the prior distribution of (b1, b2) is assumed to be bi-variate normally distributed with mean (1, 2), and 11 =var(b1), 12 = cov(b1, b2), 22 = var(b2). Note here that thelongitudinal data are very dense as daily observations wereavailable for all flies, so the model fitting should not be sen-sitive to this normality assumption as supported by the ro-bustness property discussed in previous sections.

The Cox proportional hazard regression model assumptionswere checked via martingale residuals for both data sets andwere reasonably satisfied with p-values 0.9 and 0.6, respec-tively. We thus proceeded with fitting the joint models in (1)and (2). Tables 4 and 5 summarize the EM algorithm esti-mates of together with their bootstrap means and standard

Table 5Analysis of lifetime and log-fecundity of medflies with lifetime reproduction between 300 and 749 eggs. The

bootstrap SD and the working SE, (IW

)1/2, are reported in the last two rows.

1 2 11 12 22 2e

EM estimate 0.1997 1.7618 0.1730 1.0540 0.1734 0.0301 1.4344Bootstrap mean 0.2036 1.7659 0.1739 1.0487 0.1766 0.0309 1.4333Bootstrap SD 0.0836 0.0800 0.0145 0.0784 0.0151 0.0033 0.0356(IW

)1/2 0.0672 0.0576 0.0120 0.0572 0.0112 0.0023 0.0282

deviations based on 100 bootstrap replications. The corre-sponding working SE, (IW

)1/2, is also reported on the last

row of each table for a contrast.From Tables 4 and 5, we can see that the sample bootstrap

means are close to the corresponding EM estimates, suggest-ing the feasibility of the bootstrap approach. On the otherhand, the working SE corresponding to the last step of theEM algorithm produced estimates that are noticeably smallerthan the bootstrap SD estimate. For , it yielded an estimateof 0.0801 under the setting of Table 4, which is about a 15%departure from the bootstrap SD (0.0921). The discrepancyincreases to 25% (working SE is 0.0672) for the second dataset in Table 5 due to a higher measurement error (1.4344 inTable 5 vs. 0.9883 in Table 4), consistent with the simulationfindings in Section 4. Similar discrepancies can also be seenbetween Tables 4 and 5 for all other parameters.

6. ConclusionsWe have accomplished several goals in this article: (i) Rein-forcing the merit of the joint modeling approach in WT byproviding a theoretical explanation of the robustness featuresobserved in the literature. This suggests that the likelihood-based procedure with normal random effects can be very ef-ficient and robust as long as there is rich enough informationavailable from the longitudinal data. Generally, this meansthat the longitudinal data should not be too sparse or carrytoo large measurement errors. (ii) Demonstrating the miss-ing information and implicit profile features in joint modelingboth theoretically and empirically to alert practitioners. Theefficiency loss due to missing random effects can be quite sub-stantial, as observed in numerical studies. (iii) Recommendinguntil further theoretical advances afford us reliable standarderror estimates to use the bootstrap SD estimate for estimat-ing the standard errors of the EM estimators.

However, theoretical gaps remain to validate the asymp-totic properties of the estimates in WT and the validity ofthe bootstrap SE estimates. The theory of profile likelihood


is well developed for parametric settings (Patefield, 1977), butnot immediately applicable in semiparametric settings (Mur-phy and van der Vaart, 2000; Fan and Wong, 2000). The com-plexity caused by profiling nonparametric parameters withonly implicit structure creates additional difficulties in the-oretical and computational developments for joint modeling.Further efforts will be required in future work to resolve theseissues and to provide reliable precision estimates for statisticalinference under the joint modeling setting.

Acknowledgements

The research of Jane-Ling Wang was supported in part bythe National Science Foundation and National Institutes ofHealth. The authors greatly appreciate the suggestions froma referee and an associate editor.

References

Breslow, N. E. and Clayton, D. G. (1993). Approximate infer-ence in generalized linear mixed models. Journal of theAmerican Statistical Association 88, 925.

Breslow, N. E. and Lin, X. (1995). Bias correction in gen-eralized linear mixed models with a single componentdispersion. Biometrika 82, 8191.

Carey, J. R., Liedo, P., Muller, H. G., Wang, J. L., and Chiou,J. M. (1998). Relationship of age patterns of fecundityto mortality, longevity, and lifetime reproduction in alarge cohort of Mediterranean fruit fly females. Journalof GerontologyBiological Sciences 53, 245251.

Cox, D. R. (1972). Regression models and life tables (withdiscussion). Journal of the Royal Statistical Society, SeriesB 34, 187220.

Fan, J. and Wong, W. H. (2000). Comment on On profilelikelihood by Murphy and van der Vaart. Journal of theAmerican Statistical Association 95, 468471.

Henderson, R., Diggle, P., and Doboson, A. (2000). Joint mod-eling of longitudinal measurements and event time data.Biostatistics 4, 465480.

Kiefer, J. and Wolfowitz, J. (1956). Consistency of the max-imum likelihood estimator in the presence of infinitelymany incidental parameters. Annals of MathematicalStatistics 27, 887906.

Louis, T. A. (1982). Finding the observed Fisher informationwhen using the EM algorithm. Journal of the Royal Sta-tistical Society, Series B 44, 226233.

Murphy, S. and van der Vaart, A. W. (2000). On profile like-lihood (with discussion). Journal of the American Statis-tical Association 95, 449465.

Orchard, T. and Woodbury, M. A. (1972). A missing informa-tion principle: Theory and applications. In Proceedings ofthe 6th Berkeley Symposium on Mathematical Statisticsand Probability, Volume 1, p. 697715. Berkeley: Univer-sity of California Press.

Patefield, W. M. (1977). On the maximized likelihood func-tion. Sankhya, Series B 39, 9296.

Solomon, P. J. and Cox, D. R. (1992). Nonlinear componentof variance models. Biometrika 79, 111.

Song, X., Davidian, M., and Tsiatis, A. A. (2002). A semipara-metric likelihood approach to joint modeling of longitu-dinal and time-to-event data. Biometrics 58, 742753.

Tierney, L. and Kadane, J. B. (1986). Accurate approxima-tion for posterior moments and marginal densities. Jour-nal of the American Statistical Association 81, 8286.

Tsiatis, A. A. and Davidian, M. (2004). Joint modelling oflongitudinal and time-to-event data: An overview. Sta-tistica Sinica 14, 809834.

Wulfsohn, M. S. and Tsiatis, A. A. (1997). A joint modelfor survival and longitudinal data measured with error.Biometrics 53, 330339.

Yu, M., Law, N. J., Taylor, J. M. G., and Sandler, H. M.(2004). Joint longitudinal-survival-cure models and theirapplication to prostate cancer. Statistica Sinica 14, 835862.

Received July 2004. Revised December 2005.Accepted January 2006.

Documents

Joint Modeling - Likelihood Approach Revisited