ii ,,I E L S E V I E R Statistics & Probability Letters 24 (1995) 273-279

STATISTICS& PROBABILITY

LETTERS

Second-order asymptotic efficiency of PMLE in linear models

Hua Liang Institute of Systems Science, Academia Sinica, Beijing 100080, China

Received May 1993; revised January 1994

generalized

Abstract

In this paper, the author discusses the second-order asymptotic efficiency of pseudo-maximum likelihood estimator of fl based on Yi = f ( X i , fl) + g(Ti) + el, i = 1 . . . . . n, where Xi, T i, el are independent, f ( . , - ) is known, but g is unknown, e ~ q~(') is known with mean 0 and variance a 2.

Keywords: Second-order asymptotic efficiency; Pseudo-maximum likelihood estimator; Generalized linear model

1. Introduction

Consider the model

Y = f ( X , fl) + g ( T ) + e, (1.1)

where X e R x is an explanatory variable, f l e ~ c R ~ is an unknown parameter, T is another explanatory variable, f ( . , .) is a known function, but g(.) is an unknown smooth function of T in R 1, (X, T) and e are independent, and e is the random error with mean 0 and variance a 2.

The works of second-order efficiency of asymptotically efficient estimators was originated by Fisher (1925), Rao (1961, 1962) and others in terms of loss of information, then Chibisov (1972, 1973) showed that a minimax likelihood estimator (MLE) is second-order asymptotically efficient. Pfanzagl (1973, 1975) obtained that MLE attains the second-order asymptotic efficiency. Akahira and Takeuchi systematically studied higher-order asymptotic efficiency. Liang and Cheng (1993) discussed this efficiency in the model: Y = X f l + g ( T ) + e. In this paper, we focus on the second efficiency in the model (1.1). At first we introduce concept of second-order asymptotic efficiency. Suppose that ~ is an open set of R 1. A {C,}-consistent estimator ft, is called second-order asymptotically median unibased (or second-order AMU) estimator if for any v ~ M, there exists a positive number 6 such that

lira sup C , ] P p . , { f l , < ~ f l } -½1 = 0 ;

lira sup C. I Pp,. { ft. >/fl} - ½[ = O.

0167-7152/95/$9.50 © 1995 Elsevier Science B.V. All rights reserved S S D I 0 1 6 7 - 7 1 5 2 ( 9 4 } 0 0 1 8 4 - 7

274 H. Liang / Statistics & Probability Letters 24 (1995) 2 73-2 79

Suppose 8. is a second-order AMU, Go(t, 8 ) + C,G~(t, 8) is called to be the second-order asymptotic distribution of C,(8, - 8) if

lim CnIP~.. {C.(8 . - 8) <<. t} - Go(t, 8) - C[ ~ G, (t, 8)I = O. n~ oo

As discussed below, C, is equal x/~. Let 8o ( e ~) be arbitrary but fixed. We consider the problem of testing hypothesis

Put

U H ÷: fl = fll = flo + - - ( u > 0)~--~K: fl = flo.

¢~I/2 ~n: E~o+./,/~, . ~. = ½ + ~b, is an UMP test ..

Aa..ao = {x/~(fn -- 80) ~< u}.

Then

lira P¢o+u/~.. (A~..~o) = lira PBo+u/~.~ ~ <~ flo + = ~,

so a sequence {L%.po} of the indicator of A~.,~o(n = 1, 2 . . . . ) belongs to ~ / 2 . If

~b. ~ ~1/2 n~oo

then

if

Go(t, 80) ~ H~ (t, 8o);

Go(t, 80) = H~ (t, 80),

then

G, (t, 80) ~< H ( (t, 80).

Similarly to consider next the problem of testing hypothesis

H - : 8 = 80 + ~nn(U < 0 ) ~ K : 8 = 80-

If

inf lim Epo, . (dpn)-Ho( t , 8o) ~ H ~ ( t , 8o) =0 , ~b. ~b~/2 n~oo

then

Go(t, flo) <~ Ho (t, flo);

H. Liang/ Statistics & Probability Letters 24 (1995) 273-279 275

if

Go(t,/30) = Ho (t,/30),

then

G~ (t,/30) ~< H ( (t,/30).

ft, is said to be second-order asymptotically efficient if its second-order asymptotic distribution uniformly attains the bound of the second-order asymptotic distributions of second-order AMU estimators, that is for each/3 e

Hi + (u,/3) for u > 0, Gi(u, /3) = ( Hi- (u, /3) for u < 0.

In this paper, we will construct a second-order asymptotically efficient estimator of/3 basing on P MLE of/3. Let Y, X and T be random variables such that T ranges over [0, 1], X ~ R 1 and Y is real valued. Let

{Xi, Ti, Yi, i = 1 . . . . . n} denote a sample of size n from the Yi =f(X~,/3) + g(Ti) + e~, where the errors e~ are assumed to be independent and identically distributed with mean 0 and finite variance a2. (X~, Ti) and e~ are independent. Denote

Y = ( Y 1 . . . . . Yn)', e = ( e l A .. . . e,) r,

g (T) = (ff(Tt), . . . , g(Tn))', X = (X1 . . . . , Xn)'.

The following conditons are sufficient for main results.

Condition 1. The distribution of T is absolutely continuous and its density is bounded away from 0 on [0, 1].

Condition 2. Let r and M denote nonnegative real constants 0 < r ~< 1, m is nonnegative integer such that

Lff~"~(t ')-g{m)(t)l~< M I t ' - t l r fo r0~<t , t '~<l .

Think of p = m + r as measure of the smoothness of the function g.

Condition 3. f ( x , fl) has twice continuous and bounded differentiability of/3. Denote

/'~0'2 (Y) d , ~O(.) = logtp('), I = I(tp) = j - ~ - y

S = fO"(u)tp'(u)q~(u)du, K = fE¢'(u)]3q~(u)du.

~( ') , 4)(') denote the standard normal distribution function and density function, respectively.

Condition 4. ~o(. ) is three times continuously differentiable,

iim,~o~ S supltl <~ l ~(3)(x + (t /x/~)) q~(x)dx = O.

276 H. Liang/ Statistics & Probability Letters 24 (1995) 273 279

Condition 5. J = S ¢"(u)~'(u)q~(u)du and K = ~[~b'(u)]3 ~o(u)du exist, and ~ ¢"'(u) ~o(u)du = - 3J - K holds.

Condition 6. l im, _._+~ q~(u) = l im, _~_+~ q~'(u) = lim, ~_+~ q¢'(u) = 0, Ee 4 < ~ .

2. Main results

Lemma 1 (Zhao and Bal, 1985). Suppose W1, . . . , W, are independent, and E W j = 0, 3"7=IEWf. > 0, EJ Wi[ 3 < ~ for each j, let

, / z 7 : 1 e w ? w ,

then

G.(w) = ¢ (w) + ~ q~(w)(1 - w 2)/~23/---- ~ + 0

holds uniformly for w e R 1, where ~t2 = Z~=I E W 2 , #a = Z7=1 E W 3 .

Theorem 2.1. Assume Conditions 4 - 6 hold, if for some estimator {ft, } satisfies

P t~ , . {w /n (~ . - ~) <~ u} = eb(v/-Au ) + c~(~/Au) 6~//- ~ ~ + o (2.1)

then {ft.} is second-order asymptotically efficient, where A = IE f ' Z ( X , fl), B = IE { f ' ( X , f l ) f " (X , fl)} - JE f ' 3 ( X , fi), and C = - K E f ' 3 ( X , fl).

Proof. See the p roof of Theorem 4.1.1 of Akahi ra and Takeuchi (1981). [ ]

In order to construct second asymptot ica l ly efficient est imator, we must firstly est imate 9(0. Next we introduce piecewise polynomia l to approx ima te 9. Given a positive M. , the es t imator has the form of a piecewise polynomia l of degree m based on M . intervals of length 1/M., where the (m + 1)M. coefficients are chosen by the me thod of least squares on the basis of the da ta (X1, T1, Yx) . . . . . (Xn, T . , Y.), 1 ~< i ~< n. Let I.~ = [ ( v - 1 ) / M . v / M . ] for 1 ~< v < M . and I . M . = [ 1 - 1/M., 1]. Let X.v denote the indicators function for the interval I.~, so that Z.~ = 1 or 0 according to t ~ l.v or t¢I .v . Consider the piecewise polynomial es t imator of 9 of degree m given by

Mn

O.(t) = ~ g.v(t)P.mv(t). v = l

let dm.~ be the center of the interval I.~. Let P,..v be the Tay lo r po lynomia l approx ima t ion of degree m to 0 abou t d . . . . Since

]9tm)(t ') -- 9~"~(t)] ~< Mlt ' - tl" for 0 ~< t, t' ~< 1.

There is a constant B2 > 0 such that

IZ.~(t)(P,..~ - 9(t))l <~ B 2 M ~ p for all v and t e [0, 1].

H. L iang/ Statisties & Probability Letters 24 (1995) 273-279 277

On the other hand,

M. g(T, ) - O.(T,) = g(T , ) - ~ Z. , (Ti)P, . .v(Ti)

V=I

Mn

Observe that

Z. , (Ti) = 1 v = l

= ~, X.v(T, )[g(T, ) - P,..v(T,)]. v = l

for a l l i = l . . . . ,n. So

Ig(Ti) - ~.(Ti)[ ~< B 2 M ~ p, i = 1 . . . . . n. (2.2)

Now let us explain what P M L E offl is and how we get it. By using an approximation ~.(T) to replace g ( T ) in Y = f ( X , fl) + g ( T ) + e. In this situation, according to {Yi = f ( X i , fl) + ~.(Ti) + ei, i = 1 . . . . . n} we can get MLE tiME of fl, tiME are called pseudo-MLE (PMLE) of ft. Below we will base P M L E to construct second-order asymptotically efficient estimator of ft.

Theorem 2.2. Suppose the ruth (m >1 4) cumulants o f x//nflML be less than order 1/x//-£, Conditions 1-6 hold, and lim.~ ~ n M p. = 0, then

K E f ' 3 ( X,/~ML) fl~lL = flML "t- 6ni2 E 2 f , 2 ( X , flML)

is second-order asymptotically efficient.

Remark. This theorem points that although tiME is not second-order asymptotically efficient, we can construct a second-order asymptotically efficient estimator basing on the tiME-

Proof of Theorem 2.2. It follows from (2.2) and the definition o f tiME that

,...~ ~ ' ( Y i - - f ( X i , flML) ^ ' ^ 0 = -- g (T~) ) f (X,,/3ML) i=1

= ~ ~h'(Y, - f ( X , , / 3 ) - 9 ( r i ) ) f ' ( X i , / 3 ) ] i=1

+ ~ [ d / ( Y , - f ( X , , / 3 ) - g ( , ) ) f (X,,/3) ~ " ( Y , - f ( X , , / 3 ) g(Ti))f'2(Xi,/3)J(flML--/3) T tt __

i=1

1 " + ~ , ~ [~b'"(Y, - f * ( X , , / 3 ) -- g* (T , ) ) f ' 3 (X , , / 3* ) -- 3~h"(Y~ - f ( X , , / 3 * )

, i , u - g ( T , ) ) f (X,, 13 ) f (Xi , /3*) + ¢ ' ( Y , --f(X,,/3*)

. g ( T , ) ) f (Xi, fl*)] (flmL /3)2 + O (2.3)

278 H. Liang / Statistics & Probability Letters 24 (1995) 273-279

where Ifl* - 131 ~< IflUL -- fll and [g*(T;) - g(T~)I ~< I9(T~) - ~(Ti)]. So, it is easy to verify that

( ( 1 ) ) N//n(flML -- fl) --~ N 0, I ( E f , 2 ( X ' fl)) .

T n = N//n( f lML- fl),

1 Z 0 ' ( Yi - f ( X i , fl) - g ( T i ) ) f ' ( X i , fl), Zl (fl) = ~ i=1

1 [gg'(Yi - f ( X i , [3) - g ( T i ) ) f " ( X i , fl) - O"(Yi - f (Xi, fl) z~(fl) = - ~ ,=,

-- 9 ( T i ) ) f ' 2 ( X , , fl) - I E f ' 2 ( X , fl)],

1 ~ Za(fl) = n ,~=~ [ gg"'(Y' - f ( X . fl*) - a * ( r 3 ) f ' a ( x , fl*) - 3q/'(r~ - f ( X , , f l*) - g* (T , ) ) .

i f ( X . f l * ) f " ( X i , fl*) + gg'(Yi - f ( X , , fl*) - 9" ( T i ) ) f " ' ( X i , fl*)].

By calculating, we have

EZx (fl) = EZ2 (fl) = O,

EZZ( f l ) = I E f ' 2 ( X , fl) = A,

EZ2 (fl) = E [gg'(e)f" (X, fl) -- gg "(e)f '2 (X, fl) -- A] 2,

E Z , ( fl) Z2 ( fl) = Egg '2 (e) E { f ' ( X , fl) f " (x, fl) } - Egg" (e) gg' (e) E f ' 3 ( X , fl) = B.

Then Z l ( f l ) and Z2(fl) have the asymptotically normal distribution with mean 0 and variance A and L(f l ) = EZE(f l ) , respectively, and covariance B, Meanwhile, Z3(f l ) convergence in probability to

Egg "' (e) E f '3 (X, fl) - 3Egg" (e) E { f ' (X, fl) f " (X, fl)}.

Hence it follows from (2.3) that

{ 1 } Zl ( f l )+ - A + ~ Z 2 ( f l ) 7".

Observe that

T. Zl(fl) ~0 and T. z - - A

So

Let

2w/- ~ T 2 = o.

z~(fl) --~0. A 2

z, (fl) T . = - - 4

A Z l ( f l ) Z z ( f l ) 3 B + C ( _ ~ )

w / ~ a 2 2 n v / ~ A 3 Z~( f l ) + op .

]~IL = ]~ML -- K E f ' 3 ( X ' flML) 6nI2 E Z f ' 2 ( X , flML)"

(2.4)

1t. Liang / Statistics & Probability Letters 24 (1995) 273-279

Similarly as the proof of Theorem 4.1.3 of Akahira and Takeuchi (1981), we can prove that

, - 3B+2Cx/~ld 2 _~_O(_~n) Pa,n {x//n(fl~L -- fl) ~< U} = q~(x/~U) + dp(x/Au ) 6 ~

This completes the proof of Theorem 2.2. []

279

Acknowledgements

The author is grateful to the referee for his constructive suggestions.

References

Akahira, M. and K. Takeuchi (1981), Asymptotic Efficiency of Statistical Estimators, Concepts and Higher Order Asymptotic Efficiency. Lecture Notes in Statistcs, Vol. 7 (Springer, Berlin).

Chibisov, D.M. (1972), On the normal approximation for a certain class of statistics. Proc. Sixth Berkeley Syrup. on Math. Statist. and Prob., Vol. l, pp. 153-174.

Chibisov, D.M. (1973), Asymptotic Expansions for Newman's C(~t) Tests, Proc. Second Japan-USSR Symp. on Probability Theory. Lecture Notes in Math., Vol. 330 (Springer, Berlin) pp. 16-45.

Fisher, R.A. (1925), Theory of statistical estimation, Proc. Camb. Phil. Soc., 22, 700-725. Liang, H. and P. Cheng (1993), Second order asymptotic efficiency in a partial linear model, Statist. Probab. Lett. 18, 74-83. Pfanzagl, 0973), Asymptotics expansions related to minimum contrast estimators, Ann. Statist. 1, 993-1026. Pfanzagl, (1975), On asymptotically complete classes, Proc. Summer Res. Inst. Statist. Inference Stochastic Process 2, 1 43. Rao, C.R. (1961), Asymptotic efficiency and limiting information, Proc. 4th Berkeley Syrup. on Math. Statist. and Prob., l, 531-545. Rao, C.R. (1962), Efficient estimates and optimum inference procedures in large sample, J. Roy. Statist. Soc. Set. B 24, 46-72. Zhao, L.C. and Z.D. Bai (1985), Asymptotic expansions of distribution of sum of independent random, Sci. China Set. A 8, 677-697.