Comparison of Population-Averaged and Subject-Specific

American Journal of EpidemiologyCopyright © 1998 by The Johns Hopkins University School of Hygiene and Public HealthAll rights reserved

Vol. 147, No. 7Printed in U.S.A

Comparison of Population-Averaged and Subject-Specific Approaches forAnalyzing Repeated Binary Outcomes

Frank B. Hu,1 Jack Goldberg,2 Donald Hedeker,2'3 Brian Ft. Flay,3 and Mary Ann Pentz4

Several approaches have been proposed to model binary outcomes that arise from longitudinal studies.Most of the approaches can be grouped into two classes: the population-averaged and subject-specificapproaches. The generalized estimating equations (GEE) method is commonly used to estimate population-averaged effects, while random-effects logistic models can be used to estimate subject-specific effects.However, it is not clear to many epidemiologists how these two methods relate to one another or how thesemethods relate to more traditional stratified analysis and standard logistic models. The authors address theseissues in the context of a longitudinal smoking prevention trial, the Midwestern Prevention Project. Inparticular, the authors compare results from stratified analysis, standard logistic models, conditional logisticmodels, the GEE models, and random-effects models by analyzing a binary outcome from two and sevenrepeated measurements, respectively. In the comparison, the authors focus on the interpretation of bothtime-varying and time-invariant covariates under different models. Implications of these methods for epide-miologic research are discussed. Am J Epidemiol 1998; 147:694-703.

generalized estimating equations; longitudinal studies; random-effects regression models; repeatedmeasurement

Repeated measures designs or longitudinal studiesoccupy an important role in clinical and epidemiologicresearch. In a clinical trial, for example, patients maybe randomly assigned to different treatment conditionsand repeatedly classified in terms of presence or ab-sence of clinical improvement, side effects, or specificsymptoms. Most adolescent smoking prevention trialsinvolve a baseline measure of smoking behaviors andrelated factors, followed by an intervention program,an immediate posttest, and several follow-up measure-ments. This kind of design offers the opportunity tostudy the time course of change and the long-termeffects of the treatment or intervention (1). It alsooffers increased statistical power and robustness formodel selection (2).

Received for publication February 18, 1997, and accepted forpublication September 20, 1997.

Abbreviations: Cl, confidence interval; GEE, generalized estimat-ing equations.

1 Department of Nutrition, Harvard School of Public Health, Bos-ton, MA.

2 Epidemiology and Biostatistics Division, School of PublicHealth, University of Illinois at Chicago, Chicago, IL

3 Prevention Research Center, University of Illinois at Chicago,Chicago, IL.

4 Department of Prevention Medicine, University of Southern Cal-ifornia, Los Angeles, CA.

Reprint requests to Dr. Frank B. Hu, Dept. of Nutrition, HarvardSchool of Public Health, 655 Huntington Ave., Boston, MA 02115.

Analyses of repeated measures data need to accom-modate the statistical dependence among the repeatedobservations within subjects. For normally distributeddata, variants of the random-effects model of Lairdand Ware (3) are commonly used, and software forestimating this class of models is now widely avail-able, such as SAS Proc Mixed (4), BMDP5V (5),HLM (6), and MLn (7).

Recently, several models have been proposed tomodel binary outcomes that arise from repeated mea-sures designs. Most of the models can be groupedinto two classes (8, 9): the "subject-specific" and the"population-averaged" approaches. Random-effectslogistic models (10, 11) are commonly used to esti-mate subject-specific effects, while the generalizedestimating equations (GEE) method of Liang andZeger (12) is usually used to provide population-averaged effects.

While both the GEE and random-effects approachesare extensions of models for independent observationsto time-dependent data, they address the problem oftime-dependency differently. Also, the regression co-efficients or odds ratios obtained from the two ap-proaches are numerically different, as are their inter-pretations (8, 13, 14). Although comparisons of thetwo approaches have appeared in the statistical litera-ture (9, 15-18), there has been little research on suchcomparisons that is accessible to public health re-

694

Comparison of Approaches for Analyzing Repeated Binary Outcomes 695

searchers. In addition, researchers in substantive areasare often unclear as to how these models relate tostatistical methods or models with which they arealready familiar (e.g., stratified analysis or the stan-dard logistic model). The purposes of this paper in-clude 1) describing and comparing the main featuresof random-effects logistic models and GEE logisticmodels in the context of population-averaged and sub-ject-specific approaches; 2) showing the connectionbetween these models and conventional epidemiologicmethods, such as stratified analysis and standard lo-gistic models; and 3) illustrating the use and interpre-tation of random-effects and GEE logistic models incomparison with each other and in comparison withusual logistic models by re-analyzing a longitudinalsmoking prevention dataset with multiple timepoints.

The remainder of this paper is organized as follows.We first discuss the main features of population-averaged and subject-specific approaches for binarydata. Then we describe the dataset from a smoking anddrug abuse prevention study, the Midwestern Preven-tion Project (MPP) (19). To compare models, we firstanalyze the data considering only two timepoints(baseline and one-year follow-up). Finally, we presentand compare results from standard logistic, GEE, andrandom-effects models by analyzing data from seventimepoints. We discuss the implications of these meth-ods for epidemiologic research. In the Appendix, weprovide details of the programs we used to fit the GEEand random-intercept logistic models in this paper.

THE POPULATION-AVERAGED APPROACH

To estimate treatment effects in a longitudinal pre-vention trial, we consider the following model:

PrQ* = 1)(lyty

Var(^) = H{\ - N),

(1)

(2)

where Yy denotes a binary outcome (i.e., smoking, 0 =no and 1 = yes) for subject i at time j , fiy = E{Y^denotes the expectation or the mean of the response, xt

denotes the treatment group (0 = control, 1 = treat-ment) for subject i, and ty denotes the time correspond-ing to the jth measurement for subject i. Equation 1assumes a linear relation between the log odds ofresponse and both time and treatment group; equation2 indicates that the variance of the binary response isa known function of its mean. In this model, exp(/3,)is the odds of an event at time j divided by the odds attime j — 1, controlling for treatment group, whileexp(/32) is the odds of the event among subjects in the

treatment group divided by the odds among subjects inthe control group, controlling for time. Because bothexp^,) and exp(/32) are ratios of subpopulation risk,they are referred to as population-averaged effects. Inother words, the estimate of time effects 03,) does notdistinguish between observations belonging to thesame or different subjects.

Traditional epidemiologic methods such as stratifiedanalysis (i.e., Mantel-Haenszel method (20)) and stan-dard logistic models for independent binary outcomes(21) are essentially population-averaged approaches.However, these methods are usually not appropriatefor correlated binary outcomes arising from longitudi-nal studies due to the dependency of the repeatedmeasurements. For longitudinal data, Liang and Zeger(12) proposed the generalized estimating equations(GEE) approach, which is an extension of generalizedlinear models (GLM) (22), to estimate the population-averaged estimates while accounting for the depen-dency between the repeated measurements. Specifi-cally, the dependency or correlation between repeatedmeasures is taken into account by robust estimation ofthe variances of the regression coefficients. In fact, theGEE approach treats the time dependency as a nui-sance, and a "working" correlation matrix for thevector of repeated observations from each subject isspecified to account for the dependency among therepeated observations. The "working correlation" isassumed to be the same for all subjects, reflectingaverage dependence among the repeated observationsover subjects. Several "working" correlation structurescan be specified, including independent, exchange-able, autoregressive, and unstructured. An indepen-dent working correlation assumes zero correlationsbetween repeated observations. An exchangeableworking correlation assumes uniform correlationsacross time. An autoregressive working correlationassumes that observations are only related to their ownpast values through first or higher order autoregressive(AR) process. An unstructured working correlationassumes unconstrained pairwise correlations. Liangand Zeger (12) show that under the assumption ofmissing completely at random (MCAR, discussed be-low), the GEE approach provides consistent estimatorsof the regression coefficients and of their robust vari-ances even if the assumed working correlation is mis-specified.

Estimation of the standard logistic model is equiv-alent to GEE estimation with an independent workingcorrelation structure. With repeated binary outcomes,the standard logistic model yields the same population-averaged estimates as the GEE. However, the standarderrors from the standard logistic models are biasedbecause the independence assumption is violated. The

Am J Epidemiol Vol. 147, No. 7, 1998

696 Hu et al.

biases are dependent on whether the covariates varywith time (23, 24). Regression models ignoring thetime dependency tend to overestimate the standarderrors of time-varying covariates and underestimatethe standard errors of time-invariant covariates.

THE SUBJECT-SPECIFIC APPROACH

In contrast to the population-averaged approach, thesubject-specific approach can distinguish observationsbelonging to the same or different subjects. Random-effect models are commonly used to estimate subject-specific effects. Two methods can be used to estimatethe subject-specific effects in the random-effects mod-els: maximum likelihood and conditional likelihoodprocedures (25).

For repeated binary responses, logistic regressionmodels (10, 11), probit regression models (26), andlog-linear type models (27) have been proposed. Asimple random-effects logistic model for estimatingtreatment effects in a longitudinal prevention trial canbe written as:

v.)(3)

where u, is the random subject deviation.This model is a generalization of the standard logis-

tic model in which the intercept (deviation), u(, isallowed to vary with subjects. Thus, the model is oftencalled a random-intercept logistic model. The randomeffect is usually assumed to be distributed as N (0,aj). The fixed part /30 represents the log odds of theresponse {Ytj = 1 ) for a control subject (i.e., x = 0) atbaseline (i.e., t = 0) with random effect u, = 0.

By including a random-intercept in the model, theinterdependencies among the repeated observationswithin subjects are explicitly taken into account. Thevariance-covariance structure in a random-interceptlogistic model is analogous to the "compound symme-try" form assumed in a mixed-model analysis of vari-ance (ANOVA) and the "exchangeable working cor-relation" in the GEE model described previously. Aswith the standard logistic model, likelihood ratio testsand/or Wald tests can be used to assess the treatmentor time effects. The test in the random-effects model,however, is performed while accounting for the inter-dependency among the repeated observations withinsubjects.

The interpretation of e13 for a binary independentvariable in a random-effects logistic model is some-what different from that in a standard logistic model(9). With the standard logistic model, the baseline riskis simply the proportion of positive responses in thecontrol group at baseline (e^0), while in the random-

intercept logistic model, the baseline risk is assumedto follow a distribution (eft+1i). Therefore, the corre-sponding change in absolute risk with and without thecovariate varies from one subject to another, depend-ing on the baseline rate. Consequently, the odds ratiosestimated from a random-effects logistic model addi-tionally adjust for heterogeneity of the subjects, whichcan be considered to be due to unmeasured variablessuch as genetic predisposition and unobserved influ-ences of social environmental factors. In the examplefrom the smoking prevention project that follows, wecan think of the random intercept as a subject's pro-pensity to smoke (across the study timepoints) that isindependent of the effects of the model covariates. Ina random intercept model, this propensity is assumedto be constant across time. However, by includingadditional random subject-effects into the model, thispropensity can vary across time. The unobserved pro-pensity to smoke can reflect subjects' different geneticpredispositions and/or unmeasured social environmen-tal influences particular to a given subject. For thisreason, the random effects are sometimes thought of asan omitted subject-varying covariate (9). Further dis-cussion on the philosophical detail concerning use ofrandom effects models can be found in Longford (28).

Because the estimated effects are adjusted for indi-vidual differences, they are often termed "subject-specific" effects. Therefore, the odds ratios estimatedfrom the random-effects models should be interpretedin terms of the change due to the covariates for a singleindividual (or, more specifically, individuals with thesame level on the random subject effect vt) even if thevariable is indeed a between-subjects factor such astreatment group (8). Thus, the random-effects model ismost useful when inference about individual differ-ences is of major interest.

The interpretation of a within-subject factor such astime is further complicated by missing data. For abalanced design (i.e., no missing data), the time effectsare purely within-subject effects. With missing data,however, the time effects include both between-subject and within-subject components. Generally thisdoes not impose a problem for interpretation if certainmissing data assumptions are met. As Laird (29)points out, random-effects models using maximumlikelihood estimation provide valid inferences in thepresence of ignorable non-response. By ignorable non-response, it is meant that the probability of missing-ness is dependent on measured covariates and/or pre-viously observed values of the outcome. In thissituation, the time trend for subjects with missingobservations is estimated by "borrowing" strengthfrom subjects with the same or similar characteristics.

While maximum marginal likelihood methods can



be used to estimate the parameters of the generalrandom-effects logistic model (30), for the random-intercept model, some of the parameters can be esti-mated using conditional likelihood methods (25). Toobtain a consistent estimate of the vector of coeffi-cients |3, the conditional likelihood is given by:

11,(1 + exp( - D' \ - l (4)

where D, is the within-subject difference in terms ofthe covariates. Through the conditional likelihood ap-proach, the /30 + Ui? parameters in equation 3 are re-moved by conditioning on their sufficient statistics inthe likelihood. In addition, this method only uses datafrom observations that are discordant on both responseand the covariates (e.g., time). As a result, it cannot beused to estimate the effects of a time-invarying covari-ate such as treatment condition. However, the modelcan estimate the interaction effects between time andtreatment condition. For data with only two time-points, the estimated odds ratio of time reduces to theratio of discordant pairs (31). Because conditionallikelihood is unaffected by the sampling scheme (i.e.,retrospective vs. prospective sampling), it can be usedin studies of either case-control or cohort forms (32).The random-effects approach, however, may be biasedunder case-control sampling (32).

EXAMPLE: THE MIDWESTERN PREVENTIONPROJECT

The data of interest were collected in a longitudinalsmoking and drug abuse prevention trial, the Midwest-ern Prevention Project. The design of this project andthe intervention methods have been described in detailelsewhere (19, 33). Briefly, the project is a longitudi-nal school- and community-based trial of preventionof cigarette smoking and drug abuse in adolescents.The data used in this illustration focus on a longitudi-nal panel of youth from eight schools in the KansasCity Standard Metropolitan Statistical Area who weretracked individually from 1984 to 1992. Subjects weresixth and seventh grade students who were evaluatedin a baseline measurement in Fall 1984 (n = 1,607).The subsample included in this study was 1,002 indi-viduals who were randomly selected to be followed upto adulthood (502 in the treatment group and 500 inthe control group, with assignment by school). Abouthalf of the sample were males and 82.2 percentwere whites; 40 percent were in grade 6 and 60 per-cent were in grade 7 when the program started. Self-administered questionnaires were used to assess stu-dents' smoking behaviors and related factors. Thequestionnaires were filled out by students in class-rooms prior to intervention in Fall 1984, again at 6

months, and at one year. Subsequent assessment wasconducted annually until 1992. For self-administeredsurveys, carbon monoxide measure of cigarette smok-ing, using the MiniCO Indicator (Catalyst ResearchCorporation, Owings Mills, Maryland), was used toincrease the accuracy of self-reported smoking data.

For the purpose of illustration, only one item ofseveral measurements of smoking behaviors is consid-ered, that is, smoking in the month prior to the survey(coded as 0 = no, 1 = yes; it is simply called "month-ly smoking" in the remainder of the paper). This studyfocuses on the data from the first seven timepoints ofmeasurements (from baseline to 5-year follow-up)and, for simplicity, analyzes data only for those stu-dents with complete data across all seven waves (n =682). Also, for the sake of simplicity, we ignore theintraclass correlations due to classrooms and schools(see references 34 and 35).

Table 1 treats the seven waves of measurement asrepeated cross-sectional surveys and displays the prev-alence of monthly smoking in the treatment and con-trol groups. In general, the prevalence of smokingincreased across time for both groups. However, thetrend is not the same for the two groups. For the firstfive timepoints, the rate of increase is greater for thecontrol group than for the treatment group. For the lasttwo timepoints, the prevalence leveled off in the con-trol group but continued to rise in the treatment group.

Illustrative analyses using population-averagedapproaches

To illustrate the connections between GEE andstratified analysis, we first analyze smoking data fromonly two timepoints (baseline and one year). Table 2shows the data laid out in contingency tables stratifiedby time and treatment group. The Mantel-Haenszelodds ratio for the effects of treatment group on smok-ing is 0.96 (95 percent confidence interval (CI) 0.68-1.36), indicating no significant differences in smokingbetween the two groups after controlling for time. Inthis calculation, we simply ignore the lack of indepen-dence in subjects' responses at the two timepoints. The

TABLE 1. Percent prevalence of monthly smoking in the lastmonth across time by treatment group: the MidwesternPrevention Project, 1984-1992

TimepoM Control group(n=318)

Treatment groif)(n = 364)

Baseline (T1)6 months (T2)1 year (T3)2 years (T4)3 years (T5)4 years (T6)5 years (T7)

5.76.9

15.725.531.133.034.9

9.610.211.015.120.330.235.4


698 Hu et al.

TABLE 2. Contingency tables for smoking data at baselineand one year: the Midwestern Prevention Project, 1984-1992

No

Smoking

Yes

/. Smoking and treatment group stratified by time

BaselineControlTreatmentTotal

OR* = 1.7795% Cl* 0.99-3.18

One yearControlTreatmentTotal

OR = 0.6695% Cl 0.42-1.03ORMH* = 0.9695% Cl 0.68-1.36

300329629

268324592

183553

504090

//. Smoking and time stratified by treatment groupControl group

BaselineOne yearTotal

OR = 3.1195% Cl 1.81-5.35

Treatment groupBaselineOne yearTotal

OR= 1.1695% Cl 0.72-1.87ORMH = 1.8095% Cl 1.27-2.56

300268568

329324653

185068

354075

Total

318364682

318364682

318318636

364364728

• OR, odds ratio; Cl, confidence interval; ORHH, Mantel-Haenszelodds ratio.

Mantel-Haenszel odds ratio comparing one year withbaseline smoking is 1.80 (95 percent Cl 1.27-2.58),indicating a significant increase in smoking from base-line to one year after controlling for treatment group.

The Breslow-Day test for interaction is significant (x2

= 6.95, degrees of freedom (df) = 1, p < 0.01),suggesting that the time effects are not the same for thetreatment and control groups.

We estimated two logistic models (table 3), one withonly main effects of time and treatment group and theother including a time by group interaction term sothat the results can be directly compared to those fromthe stratified analyses described above. As expected,results from the logistic models are nearly identical tothose from the stratified analyses. The interaction ef-fect of treatment group and time is significantly lessthan zero, suggesting that the increase in smoking isgreater in the control group than in the treatmentgroup. In the standard logistic regression analyses,again, the time dependency is simply ignored; therepeated observations from the same individual aretreated as independent observations.

Table 3 also lists results from GEE logistic models.While the estimates from the GEE models are similarto those from the standard logistic models, the stan-dard errors are different. The standard error for thetreatment group is smaller in the standard logisticmodel, whereas the standard errors for time and theinteraction are smaller in the GEE models. Thus, thetest statistics (Wald x2) for time and the interactionterm from the GEE models are noticeably larger thanthose from the standard logistic models.

Illustrative analyses using subject-specificapproaches

As discussed above, the random-effects approach isclosely related to the matched-pair method in epide-miology, such as the conditional likelihood approach.To illustrate this point, we lay out the joint distribu-tions of the responses for all subjects, by treatment andgroup (table 4). This setup is analogous to a matchedstudy in which the repeated observations for each

TABLE 3. Standard logistic regression and GEE* logistic regression analyzing the effects of treatmentgroup and time on smoking (baseline and one-year data): the Midwestern Prevention Project, 1984-1992

p SE* Waldx2 Odds ratio 95% Cl*Models

I. Standard logistic modelstTreatment group (0 = control, 1 = treatment)Time (0 = baseline, 1 = one year)Test for interaction

II. GEE logistic modelst,*,§Treatment group (0 « control, 1 = treatment)Time (0 - baseline, 1 » one year)Test for interaction

-0.040.59

-0.99

-0.090.59

-0.99

0.180.180.38

0.200.150.31

0.0610.56.83

0.2015.510.2

0.961.80

0.911.80

0.68-1.361.26-2.58

0.61-1.361.34-2.43

* GEE, generalized estimating equations; SE, standard error; Cl, confidence interval.t The effects of treatment group and time are estimated from the main effects model.t Exchangeable, independent, and unspecified "working" correlations give near-identical estimates and SEs.§ SEs for GEE models are robust SEs.



TABLE 4. Matched pair contingency tables for smoking dataat baseline and one year: the Midwestern Prevention Project,1984-1992

Smokingat baseline

Smoking at one year

No Yes Total

/. Joint distribution of smoking at baseline and one year

NoYesTotal

McNemar test:

95% Cl 1.53-3.82

56626

592

632790

62953

682

//. Smoking at baseline and one year stratified by treatment group

Control groupNoYesTotal

ORUm. = 6 3 3

95% Cl 2.68-14.97Treatment group

NoYesTotal

°R* , .= 1-2595% Cl 0.69-2.25

Test for treatment x timeinteractions2 =10.21, df*=

2626

268

30420

324

A, p< 0.01

381250

251540

30018

318

32935

364

* 0 R ta»- matched odds ratio for time; Cl, confidence interval; df,degrees of freedom.

subject at two timepoints can be thought of as amatched pair. Because each "pair" is matched on treat-ment group, the group effects cannot be examinedusing the matched-pair method. However, becausetime is always discordant for the two observations ineach subject, the effects of time and the interaction oftime and group can be examined using the matched-pair method.

The matched odds ratio for time is 2.42 and its 95percent confidence interval excludes one, suggesting a

significant increase in smoking from baseline to oneyear. The interaction between time and group is testedby using the "discordant pairs" for the control andtreatment groups in a single table and calculating thePearson chi-square (36). The test statistic is highlysignificant (x2 = 10.21, df = \,p< 0.01), suggestingthat the effects of time are not the same for treatmentand control groups (OR^,, = 6.33 for the controlgroup and OR^,, = 1.25 for the treatment group).

Table 5 compares the results from the conditionallogistic models and random-intercept logistic modelsby analyzing the data from baseline and one year. Theeffects of time are identical between the two methods.The random-effects logistic model is used to estimatethe effects of the treatment, controlling for time. Theestimated odds ratio for treatment group is 0.92 (95percent Cl 0.50-2.01), which can be compared with0.91 (95 percent Cl 0.61-1.36) from the GEE model.With respect to the interaction of treatment group andtime, both the estimate (absolute value) and standarderror are greater in the conditional logistic model thanin the random-effect model. But the test statistics(Wald x2) are relatively close to each other and to thatfrom the GEE logistic model.

Analyses with all seven timepoints

The standard logistic, GEE, and random-effectsmodels are now used to evaluate group differencesacross all seven timepoints from the Midwestern Pre-vention Project. In all models, the dependent variablesare the repeated measures of smoking from the6-month to 5-year follow-ups (from T2 to T7). Theindependent variables include a linear time effect(from 1 to 6), a quadratic time effect (time2), treatmentgroup, and interactions between group and the timeeffects. Sex, race, grade at baseline, and baselinesmoking are entered as controlling variables.

We estimated two models for each set of analyses,one with only main effects of sex, race, grade, baseline

TABLE 5. Conditional logistic regression and random-effects logistic regression analyzing the effectsof treatment and time on smoking (baseline and one-year data)

Models P SE* Wald x2 Odds ratio 95% Cl*

I. Conditional logistic modelsfTime (0 = baseline, 1 = one year)Test for interaction treatment group x time

II. Random-intercept logistic modelf.tTreatment group (0 = control, 1 = treatment)Time (0 = baseline, 1 = one year)Test for interaction

0.89-1.62

-0.080.89

-1.53

0.230.53

0.310.240.49

14.49.30

0.0613.79.55

2.42

0.922.42

1.53-3.82

0.50-2.011.52-3.87

* SE, standard error; Cl, confidence interval.t The effects of treatment group and time are estimated from the main effects model.X Random-effect standard deviations were estimated as oo = 2.25 (SE = 0.32) in the main effects model, and

o = 2.38 (SE - 0.35) in the model with interaction.


700 Hu et al.

smoking, time, and treatment group and the other thatadded time2, time by group, and time2 by group.

Table 6 shows the estimates, standard errors, andWald x2 from all models. In all main-effects models,the linear time effect is significantly larger than zero,indicating an overall increasing trend of smokingacross time. The effect of treatment group is signifi-cantly less than zero, indicating increased smoking inthe control group compared with the treatment groupafter controlling for time and other covariates. In ad-dition, the race effect is significant, indicating thatwhites are more likely to smoke than nonwhites.Those who smoked at baseline are more likely tosmoke at later timepoints.

In the models including interaction terms, the qua-dratic time effects are significantly less than zero,indicating that in the control group (i.e., group = 0),the rate of increase in smoking has leveled off. Thegroup by linear effects of time are significantly smallerthan zero, suggesting that the increasing rate is greaterin the control group than in the treatment group. Thegroup by quadratic time effects, however, are signifi-cantly larger than zero. Taken together, there is adeceleration in the increasing trend of monthly smok-ing in the control group (estimate of the quadratictrend is —0.16 from the random-intercepts model) anda slight acceleration in the treatment group (estimateof the quadratic trend is -0.16 + 0.20 = 0.04 fromthe random-intercepts model).

In general, the parameter estimates from the stan-dard logistic model and the GEE model are relativelyclose, whereas the Wald •£ from the random-effectsmodel and the GEE model are relatively close. Asexpected, the standard errors for time-invariant covari-ates such as sex, race, and treatment group are smallerin the standard logistic models, while the standard

errors for time-varying covariates such as linear trendand the interaction terms are generally smaller in theGEE models. Both the estimates and standard errorsfrom the random-effects model are larger than thosefrom the GEE models, although the test statistics arerelatively close.

DISCUSSION

In this paper, we have illustrated how GEE modelsrelate to stratified analyses and standard logistic mod-els and how random-effects models relate to thematched-pair method in epidemiology. Understandingthese connections might be helpful in making a choiceof methods for correlated outcomes as well as ininterpreting estimated effects from different models.The GEE approach models the marginal distributionsand treats the longitudinal data as though they werecross-sectional. The interpretation of estimated regres-sion coefficients and odds ratios is in line with theMantel-Haenszel method and the standard logisticmodels. The dependence between repeated observa-tions is taken into account by robust variance estima-tion. When the number of subjects is large and missingdata are not an issue (i.e., no missing or MCAR,discussed below), the estimated regression coefficientsfrom the standard logistic model should be very sim-ilar to the estimates from the GEE method (37). How-ever, the standard errors from the standard logisticmodels are biased. The biases are dependent onwhether the covariates vary with time. Consistent withthe statistical literature (23, 24), our results show thatregression models ignoring the time dependency tendto overestimate the standard errors of time-varyingcovariates and underestimate the standard errors oftime-invariant covariates.

TABLE 6. Logistic models estimating intervention effects on the trends of smoking across all follow-up timepoints: theMidwestern Prevention Project, 1984-1992

KfUVartcUO

Main effects modelSex (0 = female. 1 = mate)Race (0 = nonwhite, 1 = white)Grade (0 = 6th. 1 = 7th)BaseBne smoking (0 = no, 1 = yes)Time (1-6)Group (0 = control, 1 = treatment)

Interaction terms addedTime!Group x«meGroup x time2

Standard logisticmodel

Estimate SEt

0.15 0.081.13"* 0.14

-0.011.92"0.37"

-0 .47"

- 0 . 1 1 "-0 .93"0.13"

0.10• 0.13• 0.03• 0.10

• 0.02• 0.26* 0.03

Waldx2

3.3767.60.01

216.68213.92

22.96

20.2712541457

Estimate

0.171.24'"0.031.92***0.37«"

-O.43«»

- 0 . 1 1 " *-0.94***0.13»»*

GEEt logisticmodel

RobustSE

0.130240.160220.020.16

0.020220.03

Waldx2

1.7926.510.04

76.51216.85

7.42

30.1418262156

Random-interceptlogistic model*

Estimate

0241.72**'0.013 .00" '0 5 5 * "

- 0 . 6 7 ' "

-0 .16*"-1 .40*"

020"*

SE

0.190290230.340.030.23

0.030.320.04

Waldx2

1.5636.480

76.56419.02

8.53

26.3219.0121.07

• p < 0.05; • • p < 0.01; • • • p < 0.001.t GEE, generalized estimating equations; SE, standard error.t Random-effect standard deviations were estimated as oo = 1.96 (SE = 0.12) in trie main effects model, and oo

interaction terms.= 1.99 (SE = 0.13) In the model wth



Although the random-effects approach naturally re-lates to the conditional logistic model with respect tothe interpretation of the estimated regression coeffi-cients, these two approaches are distinct in handlingthe random-intercept. While the random-effects ap-proach using maximum marginal likelihood estimationexplicitly models and estimates the random subjecteffects (i.e., the degree of heterogeneity across sub-jects in the probability of response, not attributable tocovariates in the model), the conditional likelihoodapproach treats random subject effects as "nuisance"and they are conditioned out of the likelihood func-tion. In addition, the estimated covariate effects in themore general random-effects logistic model are basedon both within-subject and between-subject compari-sons, while the conditional likelihood approach isbased entirely on within-subject comparisons and thusprovides no information about covariates which do notvary over time. Under a restricted condition (no miss-ing data and no between-subject components), theconditional likelihood approach is as efficient as therandom-effects approach (25). As such, the condi-tional likelihood approach is appropriate for longitu-dinal analyses if only within-subject effects are ofinterest.

For the purposes of illustration, we only analyzedsubjects with complete data. In reality, missing dataare inevitable in longitudinal studies. In order for themodels to provide valid estimates in the presence ofmissing data, certain assumptions have to be met (38).The most stringent assumption is data missing com-pletely at random (MCAR). Under this assumption,the missingness does not depend on any individualcharacteristics. In other words, missing subjects can beconsidered as a simple random sample of all subjects.However, MCAR can also be satisfied if missingnessis explained by model covariates, e.g., treatment con-ditions, age, sex, and race. This special case of MCARis what Little (39) refers to as covariate-dependentmissing. The GEE model discussed in this paper re-quires this assumption. The assumption for random-effects models is less restrictive, namely, the missingdata are assumed to be missing at random (MAR),which conditionally allows missingness to depend onan individual's previously observed values of the de-pendent variable. This assumption is more realistic inpractice.

The absolute values of the estimates from therandom-effects models are generally larger than thosefrom GEE models (9). The discrepancies in the estimatesbetween the two approaches are largely dependent on thecorrelation between the repeated measures. However,the "marginal" coefficients of fitted random-effectsmodels are very similar to those from a GEE model. In

fact, a subject-specific estimate can be converted to amarginal or population-averaged coefficient throughthe following equation (8):

/3PA - 0.346<x2J, (5)

where /3PA is a population-averaged coefficient, /3SS isa subject-specific estimate, and cr2

v is the variance ofthe random effect. This relation can be shown usingthe estimates obtained from the random-effects andGEE logistic models listed in table 6.

A relevant question is which approach is more ap-propriate (i.e., GEE or random-effects). GEE modelsare desirable when the research focus is on differen-ces in population-averaged response, while random-effects models are appropriate when the research focusis on the change in individuals' responses (8, 25). Inour example, if the differences in the increasing trendsof smoking between treatment and control groups areof interest, then random-effects models estimating thechanges in individuals' smoking behavior across timeare more appropriate. The random-effect approach canbe also useful in modeling co-twin control studies,where the major interest is the within-pair compari-sons across exposure discordant pairs (40). In thissituation, the random-intercept reflects the depen-dency within each twin pair over and above the influ-ence of other model terms. The heterogeneity in thesepair-varying intercepts may be due to unmeasuredshared genetic and environmental factors common tothe twins within a pair. On the other hand, the GEEapproach is more desirable when the objective is tomake inference about group differences. In the smok-ing prevention example, if we want to estimate theaveraged treatment effect, regardless of individualchange over time, then the population-averaged pa-rameter is of more interest. An advantage of the GEEapproach is that it can provide robust variance estima-tion, whereas the random-effects may be sensitive todifferent assumptions about the variance and covari-ance correlation structure, which are usually difficultto validate (8).

In epidemiology, longitudinal studies have beenused in many situations, such as clinical and preven-tion trials, prospective studies of exposure-disease re-lations, and repeated health services utilization sur-veys. In these situations, different types of measuresover time may emerge, such as repeated measures ofcontinuous variables (e.g., weight in pounds (kg),blood pressure in mmHg), repeated measures of binaryvariables (e.g., obesity, hospitalization), repeated mea-sures of ordinal variables (e.g., frequency or severityof a symptom), and other measures that are irrevers-ible over time (e.g., death). Software for the analysisof repeated measures of continuous variables is now


702 Hu et al.

widely available, such as Proc Mixed in SAS andBMDP5V. Survival analyses are widely used to ana-lyze the timing of "absorbing" events such as death oronset of a disease in cohort studies.

The methods discussed in this paper are suitable forthe analyses of repeated measures of binary variablesin epidemiology. In many rich data bases of longitu-dinal studies in epidemiology, such as the Framing-ham Study (41) or the Nurses' Health Study (42),repeated measures of discrete outcomes are common.Traditional analyses of these longitudinal studies haveoften been restricted to data obtained from baselineand one other timepoint. The statistical models dis-cussed in this paper, while more complex than thetraditional approaches, use all available data and canproduce more efficient estimates (2, 43).

ACKNOWLEDGMENTS

Data collection for this project was supported by NationalInstitute of Drug Abuse (NIDA) grant no. 03976 and dataanalysis was supported by NIDA grant no. 06307. Thisresearch was conducted when Dr. Hu was at PreventionResearch Center, University of Illinois at Chicago.

The authors thank Drs. Dick Campbell and Paul Levy formany helpful suggestions. They also thank two anonymousreferees for helpful comments.

REFERENCES

1. Bock RD. Multivariate statistical methods in behavioral re-search. New York: McGraw Hill, 1975.

2. Zeger SL, Liang K-Y. An overview of methods for the anal-ysis of longitudinal data. Stat Med 1992;11:1825-39.

3. Laird NM, Ware JH. Random effects models for longitudinaldata. Biometrics 1982;38:963-74.

4. SAS. "The mixed procedure." In: SAS/STAT software:changes and enhancements, release 6.07. (SAS technical re-port no. P-229). Cary, NC: SAS Institute, Inc, 1992:chapter16.

5. Dixon WJ. BMDP statistical software. Berkeley, CA: Univer-sity of California Press, 1986.

6. Bryk AS, Raudenbush SW, Congdon RT. Hierarchical linearmodeling with the HLM/2L and HLM/3L programs. Chicago,IL: Scientific Software, Inc, 1994.

7. Woodhouse G. A guide to MLn for new users. London:Multilevel Models Project, Institute of Education, Universityof London, 1995.

8. Zeger SL, Liang K-Y, Albert PS. Methods for longitudinaldata: a generalized estimating equation approach. Biometrics1988;44:1049-60.

9. Neuhaus JM, Kalbfleisch JD, Hauck WW. A comparison ofcluster-specific and population-averaged approaches for ana-lyzing correlated binary data. Int Stat Rev 1991;59:25-36.

10. Stiratelli R, Laird NM, Ware JH. Random-effects models forserial observations with binary response. Biometrics 1984;40:961-71.

11. Wong GY, Mason WM. The hierarchical logistic regressionmodel for multilevel analysis. J Am Stat Assoc 1985;80:513-24.

12. Liang K-Y, Zeger SL. Longitudinal data analysis using gen-

eralized linear models. Biometrika 1986;73:13-22.13. Galbraith JI. The interpretation a regression coefficient. Bio-

metrics 1991;47:1593-6.14. Zeger SL, Liang K-Y, Albert PS. Response: The interpretation

of a regression coefficient. Biometrics 1991;47:1596-7.15. Park T. A comparison of the generalized estimating equation

approach with the maximum likelihood approach for repeatedmeasurements. Stat Med 1993;12:1723-32.

16. Ten Have TR, Landis JR, Weaver SL. Association models forperiodontal disease progression: a comparison of methods forclustered binary data. Stat Med 1995; 14:423-9.

17. Newhaus JM. Statistical methods for longitudinal and clus-tered designs with binary responses. Stat Methods Med Res1992;l:249-73.

18. Pendergast JF, Gange SJ, Newton MA, et al. A survey ofmethods for analyzing clustered binary response data. Int StatRev 1996;64:89-118.

19. Pentz MA, Dwyer JH, MacKinnon DP, et al. A multicommu-nity trial for primary prevention of adolescent drug use.JAMA 1989;261:3259-66.

20. Mantel N, Haenszel W. Statistical aspects of the analysis ofdata from retrospective studies of disease. JNCI 1959;22:719-48.

21. Hosmer DW Jr, Lemeshow S. Applied logistic regression.New York: John Wiley & Sons, Inc, 1989.

22. McCullagh P, Nelder JA. Generalized linear models. 2nd ed.New York: Chapman and Hall, 1989.

23. Dunlop DD. Regression for longitudinal data: a bridge fromleast squares regression. Am Stat 1994;48:299-303.

24. Fitzmaurice GM, Laird NM, Rotnitzky AG. Regression mod-els for discrete longitudinal responses. Stat Sci 1993;8:284-309.

25. Diggle P, Liang K-Y, Zeger SL. Analysis of longitudinal data.New York: Oxford University Press, 1994.

26. Gibbons RD, Bock RD. Trend in correlated proportions. Psy-chometrika 1987;52:113-24.

27. Goldstein H. Nonlinear multilevel models, with an applicationto discrete response data. Biometrika 1991;78:45-51.

28. Longford NT. Random coefficient models. New York: OxfordUniversity Press, 1993.

29. Laird NM. Missing data in longitudinal studies. Stat Med1988;7:305-15.

30. Hedeker D, Gibbons RD. A random-effects ordinal regressionmodel for multilevel analysis. Biometrics 1994;50:933-44.

31. McNemar Q. Note on the sampling error of the differencebetween correlated proportions or percentages. Psychometrika1947;12:153-7.

32. Neuhaus JM, Jewell NP. The effect of retrospective samplingon binary regression models for clustered data. Biometrics1990;46:977-90.

33. Pentz MA, MacKinnon, Flay BR, et al. Primary prevention ofchronic diseases in adolescence: effects of the MidwesternPrevention Project on tobacco use. Am J Epidemiol 1989; 130:713-24.

34. Siddiqui O, Hedeker D, Flay BR, et al. Intraclass correlationestimates in a school-based smoking prevention study: out-come and mediating variables, by sex and ethnicity. Am JEpidemiol 1996; 144:425-33.

35. Murray DM, Rooney BL, Hannan PJ, et al. Intraclass corre-lation among common measures of adolescent smoking: esti-mates, correlates, and applications in smoking preventionstudies. Am J Epidemiol 1994;140:1038-50.

36. Breslow NE, Day NE. Statistical methods in cancer research.Vol 1. The analysis of case-control studies. (IARC ScientificPublications no. 32). Lyon: International Agency for Researchon Cancer, 1980.

37. Zeger SL. Commentary. Stat Med 1988;7:161-8.38. Little RJA, Rubin DB. Statistical analysis with missing data.

New York: Wiley, 1987.39. Little RJA. Modeling the drop-out mechanism in repeated-

measures studies. J Am Stat Assoc 1995;90:1112-21.40. Hu FB, Goldberg J, Hedeker D, et al. Modeling ordinal



responses from co-twin control studies. Stat Med. In press.41. Dawber TR. The Framingham Study. Cambridge, MA: Har-

vard University Press, 1980.42. Willett WC, Green A, Stampfer MJ, et al. Relative and abso-

lute excess risks of coronary heart disease among women whosmoke cigarettes. N Engl J Med 1987;317:1303-9.

43. Dwyer J, Feinleib M. Introduction to statistical models forlongitudinal observation. In: Dwyer JH, Feinleib M, Lippert P,et al, eds. Statistical models for longitudinal studies of health.New York: Oxford University Press, 1992:3-48.

44. Karim MR, Zeger SL. GEE: a SAS macro for longitudinaldata analysis. (Technical report no. 674). Baltimore, MD:Department of Biostatistics, John Hopkins University, 1988.

45. Davis SD. A computer program for regression of repeatedmeasures using generalized estimating equations. ComputMethods Programs Biomed 1993 ;40:15-31.

46. SAS Institute Inc. SAS/STAT software: the GENMOD pro-cedure, release 6.09. (SAS technical report no. P-243). Cary,NC: SAS Institute Inc, 1993.

47. Hedeker D, Gibbons RD. MIXOR: a computer program formixed-effects ordinal regression analysis. Comput MethodsPrograms Biomed 1996;49:157-76.

48. Statistics and Epidemiologic Research Corp. EGRET refer-ence manual. Seattle, WA: SERC, 1992.

49. SAS Institute Inc. SAS/STAT software: the PHREG proce-dure. Cary, NC: SAS Institute Inc, 1993.

APPENDIX

The GEE logistic models in this paper were fit using aSAS macro written by Karim and Zeger (44). The solutionto the GEE involves an iterative solution that alternatesbetween quasi-likelihood methods (using iteratively re-weighed least-squares) for estimating parameter estimatesand a robust method for estimating correlation as a functionof the parameter estimates. Davis (45) has also published a

FORTRAN 77 program to fit the GEE models. Besidesfitting logistic models, both the Karim and Zeger programand the Davis program can be used to fit linear models forcontinuous outcomes and Poisson models for count data.The specification of the link and variance functions is thesame as in fitting the generalized linear models (GLM) byGENMOD procedure in SAS 6.08 (46). The GENMODprocedure in SAS 6.12 can now fit GEE models.

The random-intercept logistic model in this paper wasfitted by a FORTRAN program called MIXOR written byHedeker and Gibbons (47). MIXOR can allow for multiplerandom-effects (e.g., random intercept and random slope)and includes both logistic and probit response functions.Because the likelihood does not exist in a closed form forthe general random-effect model, Gauss-Hermite quadra-ture method is used to perform the integration numericallyby summing over a specified number of quadrature points.Usually, the more points used, the more accurate the ap-proximation, but the more time it takes. In the present paper,we used 20 quadrature points for model estimation. Inaddition, MIXOR can be used to fit random-effects modelsfor ordinal response data (30).

The random-intercepts model described in this paper canalso be fit by EGRET (48). The random-intercept model isreferred to as the logistic-normal model by EGRET. As withMIXOR, Gauss-Hermite quadrature is used to integrateover the random-effects distribution.

Besides the logistic-normal model, EGRET can also fitbeta-binomial logistic models and logistic-binomial models.When using the same number of quadrature points (20points), MIXOR and EGRET produce almost identical re-sults for the data described above. The conditional logisticmodels in this paper were estimated using the SAS PHREGprocedure (49). EGRET can also fit conditional logisticmodels.


Documents

Comparison of Population-Averaged and Subject-Specific