1

NO

NPA

RA

ME

TR

ICR

AN

DO

ME

FF

EC

TS

MO

DE

LSA

ND

LIKE

LIHO

OD

RA

TIO

TE

ST

S

Oct11,2002

David

Ruppert

CornellU

niversity

ww

w.orie.cornell.edu/̃

davidr

(These

transparenciesand

preprintsavailable

—

linkto

“RecentTalks”

and“R

ecentPapers”)

Work

donejointly

with

Ciprian

Crainiceanu,C

ornellUniversity

2

OU

TLIN

E�

Sm

oothingcan

bedone

usingstandard

mixed

models

software

because

�

Splines

canbe

viewed

asB

LUP

sin

mixed

models

�

This

random-effects

splinem

odelextendsto:

�

Sem

iparametric

models

(allows

parametric

submodels)

�

Longitudinaldata

�

nestedfam

iliesofcurves

3

EX

AM

PLE

�R

ickC

anfieldand

Chuck

Henderson,

Jr.at

Cornell

are

working

oneffects

oflow

-levellead

exposureon

IQof

children.

�

They

havea

mixed

model

butthe

dose-responsecurve

shouldbe

modeled

nonparametrically.

4

EX

AM

PLE

—C

ON

T�

They

askedS

AS

isa

”PR

OC

GA

MM

IXE

D”w

ouldbe

avail-

ablesom

eday.

�

shortanswer

was

”no”

�

Then,

theyfound

Matt

Wand’s

work

andthen

contacted

me.

�

Now

theyknow

thatGA

MM

IXE

D�

GLM

MIX

ED

.

�

SA

Shas

GA

MM

IXE

Dand

doesnotknow

it!

5

TE

ST

ING

INT

HIS

FR

AM

EW

OR

K�

Inprinciple,likelihood

ratiotests

(LRT

s)could

beused

to

testforeffects

ofinterest

�

E.g.,hypothesis

thatacurve

islinearorthatan

effectis

zero��

avariance

component

(andpossibly

afixed

effect)is

zero

�

allows

anelegant,unified

theory

6

TE

ST

ING

—C

ON

T�

How

ever,thedistribution

theoryofLR

Ts

iscom

plex:�

thenullhypothesis

ison

theboundary

ofthe

parame-

terspace,

so“standard

theory”suggests

chi-squared

mixtures

asthe

asympototic

distribution.

�

butstandardasym

ptoticsdo

notapplybecause

ofcor-

relation

�

forthe

caseofone

variancecom

ponent,we

nowhave

asymptotics

thatdoapply

7

UN

IVA

RIA

TE

NO

NPA

RA

ME

TR

ICR

EG

RE

SS

ION

�m

odel

��������� ��

�

letting�

bea

spline

� � �����

� �� �

�� �� ��

���� ��

��������� ��

willbe

treatedas

“randomeffects”

�

assume

theyare

iid�! �#"$% �

�

sizeof"

$%

controlsthe

amountofshrinkage

orsm

ooth-

ing.

8

NO

NPA

RA

ME

TR

ICM

OD

ELS

FO

RLO

NG

ITU

DIN

AL

DA

TA���&

is' thobservation

on( thsubject

�

considerthe

nonparametric

model

��& �� �& � �� �& � ��&

�

modelthe

“population”curve�

asa

spline:

� � �����

� �� �

�� �� ��

���� ��

�

modelthe

“( thsubject”

curve�� asanother

spline:

�� � ��� �

� )* �+� �

�� �� � * �+�

���� ��

9

PO

PU

LAT

ION

CU

RV

E�

Recall:

� � �����

� �� �

�� �� ��

���� ��

��� ��������

willbe

treatedas

“fixedeffects”

��������� ��

willbe

treatedas

“randomeffects”

�

assume

theyare

iid�! �#"$%,.- �

(/

=“population”)

�

thisassum

ptioncan

beview

edas

aB

ayesianm

odel

�

somew

hatdifferentthatusualinterpretationofrandom

effects

10

SU

BJE

CT

CU

RV

ES

�R

ecall:

�� � ��� �

� )* �+� �

�� �� � * �+�

���� ��

�)* �+� �����

� )* �+�

willbe

treatedas

“randomeffects”

�

assume

theyare

iid�! �#"$0 �

�

thisis

atypical“random

effects”assum

ption

�� * �+� ������ � * �+�

willalso

betreated

as“random

effects”

�

assume

theyare

iid�! �#"$%,21 �

(3

=“subject”)

11

NU

LLH

YP

OT

HE

SE

SO

FIN

TE

RE

ST

�R

ecall:

�� � ��� �

� )* �+� �

�� �� � * �+�

���� ��

�"$0 �"$%,21 �

��

nosubjecteffects

�"$%,21 �

��

subjecteffectsare4 th

degreepolynom

ials

12

RE

LAT

ED

WO

RK

�B

rumback

andR

ice(1998)

�

Zhang,Lin,R

az,andS

owers

(1998)

�

Linand

Zhang

(1999)

�

Rice

andW

u(2001)

See

referencesatend.

13

BA

LAN

CE

1-WA

YA

NO

VA

�m

odel:5�& �6 �� ��& �( �7�������8

and' �7������:9 �

and

���;� �#"$% �

�

nullhypothesis:

<�>="$% �

�

14

�If8

?@

with9

fixed,then

�ABCDFEG� ?�$IH$� �$ H$� �

(Selfand

Liang,1987;Stram

andLee,1994)

�

This

isthe

iidcase

ifwe

takethe

subjectsas

“observa-

tions”

Note:

The

equivalentfixed

effectshypothesis

is�� �JJJ�

�K �

.

�

Then

theLR

testisequivalentto

theF

-test

� �ABCD EG� ?H$K�L

�

under<�

15

�If9

?@

with8

fixed,then

�ABCD EG� �8

MONPK L

� �7 �BCD NPK L

� �QORS>T

UVXWY[Z�\ �

and

�ABCD G EG� � 8

�7�] NK L� �7 �BCD NK L��^ R

S TVXWY Z �\ �

where N

K L� ;

_a`VXWYK L�

and NPK�L

� ;_a`V WYK �

(Crainiceanu

andR

uppert,2002)

16

010

2030

4050

6070

8090

1000.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

Num

ber of levels

ProbabilityM

LR

EM

L

1-Way

AN

OV

A:Bb

cdef/gh ] BCDFEG� � ^

17

Pinheiro

andB

ates(2000,p.87)

�sim

ulatedthe

LRT

�

foundsom

eem

piricalevidencethatthe�iH$� �

iH$� m

ixture

isbetter

replacedby4� H

$� 7 �4�� H $�

for4�kj �i .

These

theoreticalresultshelp

explaintheir

findings.

18

03

90 3 9

QQ

−plot for χ21 versus the distributions in equation (9), K

=3, 5, 20

Q0.99 =6.63

6.63

5.70

20

5

3

1-Way

AN

OV

A:asym

pt.nulldistofR

LR,given

RLRj

0

19

03

90 3 9

QQ

−plot for χ21 versus the distributions in equation (10), K

=3, 5, 20

Q0.99 =6.63

6.63

4.99

20

5

3

1-Way

AN

OV

A:asym

pt.nulldistofLR

,givenLRj

0

20

PE

NA

LIZE

DS

PLIN

ES�

model:

��l�m

�� �� �

�

nullhypothesis:

<�>=m

�� ��� �� ��� �� � � Ln � L

n�op �

�

alternativehypothesis:

<q =m � ��� �� ��� �� �

����� ���

���� �� �

21

�notation:

r �s �� �������� � �������� ��t u

�

penalizedleastsquares:

minim

ized� �� ] �� �m

� v r�^ $ wr uxr�

with

x �yzz{

| L� }~~� �

22

�sam

eas

BLU

Pin

alinear

mixed

modelw

ith

Cov��� �"

$% |

and

w � "$�

"$%

(Brum

back,Ruppert,and

Wand,1999)

23

�new

formofnull:

ifo �

"$% �

or,ifo j

,�� Ln � � �

JJJ��� �

and"$% �

�

24

Exam

ple:(C

rainiceanuand

Ruppert,2002)

�� ’sequally

spaced

�

20equally

spacedknots

�4 �o�

(constantm

eanversus

piecewise

constant

mean)

Then,

Bbcde

f/gh ] BCD G EG� � ^ ���i���

not.5

and

Bbcde

f/gh ] BCD EG� � ^ ���i �i�

not.5

25

010

2030

4050

6070

8090

1000.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95 1

Num

ber of knots

ProbabilityM

LR

EM

L

P-splines: Bb

cdef/gh ] BCD EG� � ^

26

OR

TH

OG

ON

ALIZ

AT

ION

�one

canapply

Gram

-Schm

idttothe

“designm

atrix”�

power

functionsare

replacedby

orthogonalpolynomi-

als

�

“Plus

functions”are

replacedby

splinebasis

functions

thatareorthogonalto

polynomials

�

The

asymptotics

oftheLR

Tare

changedby

thisreparam

etriza-

tion

27

�A

symptotics

areessentially

thesam

eas

for1-way

AN

OV

A

with

�8

( �#

levels)= �

( �

#knots)

+1

�

E.g.,5

levelsis

like4

knots

28

010

2030

4050

6070

8090

1000.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95 1

Num

ber of knots

Probability

ML

RE

ML

P-splines

010

2030

4050

6070

8090

1000.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

Num

ber of levels

Probability

ML

RE

ML

Orthogonalized

=1-w

ayA

NO

VA

Asym

ptoticnullprobabilities

thatlog-LRis

zero

29

Quantiles of the asym

ptotic distribution (n=∞

)

Quantiles of distributions

n=50n=100n= ∞0.5:0.5 m

ixture

q0.66

q0.95

q0.99

q0.995

1.74

4.20

5.32 0

Com

parisonoffinite-sam

pleand

asymptotic

quantiles

Hypotheses:

lineartrend

versus20-knotlinear

spline

30

Com

parisonofLR

Tw

ithothertests

Reference:

Crainiceanu,

Ruppert,

Aerts,

Claeskens,

and

Wand

(2002,inpreparation)

�

Results

innexttable

arefor

testing

�

constantmean

versus

�

generalalternative

�

piecewise

constantspline,or

�

linearspline

31

�T

hecom

parisonsare

made

with

an

�increasing,

�

concave,and

�

periodic

mean

function,chosenso

thatgoodtests

hadpow

erap-

proximately

0.8

32

�R

-testisfrom

Cantoniand

Hastie

(2002)�

F-testis

asin

Hastie

andT

ibshirani(1990)

�

“C”

means

alternativeis

apiecew

iseconstantfunction

�

“L”m

eansalternative

isa

linearspline

33

�“1”

means

estimate

underalternative

hasD

Fone

greater

thanunder

null

�

“ML”m

eanssm

oothingparam

eterunderalternativeis

cho-

senby

ML

�

“GC

V”

means

smoothing

parameter

underalternative

is

chosenby

GC

V

34

TestA

veragepow

erM

aximum

power

Minim

umP

ower

RLR

T-C0.8885

0.96600.8166

R-G

CV

-L0.8737

0.99100.7188

R-M

L-C0.8615

0.99160.7022

F-M

L-L0.8569

0.87960.8328

R-M

L-L0.8569

0.87960.8328

F-M

L-C0.8534

0.99280.6708

F-G

CV

-L0.8482

0.99460.6634

LRT-L

0.75610.8466

0.6832

F-1-C

0.70870.8442

0.4816

F-1-L

0.67750.9414

0.3012

R-1-L

0.62390.9126

0.1462

R-G

CV

-C0.6144

0.92840.3392

35

Conclusions

�S

tandardasym

ptoticsare,in

general,notsuitable

�

Better

asymptotics

forone

variancecom

ponentare

fea-

sible

�

For

more

thanone

variancecom

ponent,onem

ightneed

touse

simulation

togetp-values

36

References

Brum

back,B

.,and

Rice,

J.,(1998),

Sm

oothingspline

models

forthe

analsysiof

nestedand

crossedsam

plesofcurves,JA

SA

,93,944–961

Brum

back,B

.,R

uppert,D

.,and

Wand,

M.P.,

(1999).C

omm

enton

“Variable

selec-

tionand

functionestim

ationin

additivenonparam

etricregression

usingdata-based

prior”by

Shively,K

ohn,andW

ood,JAS

A,94,794–797.

Cantoni,E

.,and

Hastie,T.J.,2002.

Degrees

offreedom

testsfor

smoothing

splines,

Biom

etrika,89.251–263.

Crainiceanu,C

.M.,and

Ruppert,D

.,(2002),Asym

ptoticdistribution

oflikelihoodratio

testsin

linearm

ixedm

odels,submitted

Crainiceanu,

C.

M.,

Ruppert,

D.,

andV

ogelsang,T.

J.,(2002).

Probability

thatthe

mle

ofa

variancecom

ponentis

zerow

ithapplications

tolikelihood

ratioTests,

manuscript

37

Crainiceanu,

C.

M.,

Ruppert,

D.,

Aerts,

M.,

Claeskens,

G.,

andW

and,M

.,(2002),

Testsofrolynom

ialregressionagainsta

generalalternative,inpreparation.

Hastie,

T.J.,T

ibshirani,R

.,1990.

Generalized

Additive

Models,

London:C

hapman

andH

all.

Lin,X

.,and

Zhang,

D.

(1999),Inference

ingeneralized

additivem

ixedm

odelsby

usingsm

oothingsplines,JR

SS

-B,61,381–400.

Pinheiro,J.,and

Bates,D

.,(2000),Mixed-E

ffectsM

odelsin

Sand

S-P

LUS

,Springer,

New

York.

Rice,J.,and

Wu,C

.,(2001),Nonparam

etricm

ixedeffects

models

forunequally

sam-

plednoisy

curves,”B

iometrics,57,253–259.

Self,S

.,andLiang,K

.,(1987).A

symptotic

propertiesofm

aximum

likelihoodestim

a-

torsand

likelihoodratio

testsunder

non-standardconditions,JA

SA

,82,605–610.

38

Stram

,D

.,Lee,

J.,(1994).

“Variance

Com

ponentsTesting

inthe

LongitudinalMixed

Effects

Model,”

Biom

etrics,50,1171–1177.

Zhang,D

.,Lin,X.,R

az,J.,andS

owers,M

.(1998),Sem

i-parametric

stochasticm

ixed

models

forlongitudinaldata,JA

SA

,93,710–719