Z. D. Bai. and. Y. Wu. Department of Mathematics and Statistics, York University, 4700 Keele Street, North York, Ontario, Canada M3J 1P3

Transkript

1 joural of multivariate aalysis 63, (1997) article o. MV Geeral M-Estimatio Z. D. Bai Departmet of Applied Mathematics, Natioal Su Yat-se Uiversity, Kaohsiug, Taiwa ad Y. Wu Departmet of Mathematics ad Statistics, York Uiversity, 4700 Keele Street, North York, Otario, Caada M3J 1P3 I this paper, a geeral form of M-estimatio is proposed ad some asymptotics are ivestigated. The model covers all liear ad oliear regressio models, AR time series, EIVR models, etc. as its special cases. The dispersio fuctios may be covex or differeces of covex fuctios, the later covers almost all useful choices of the dispersio fuctios Academic Press 1. INTRODUCTION I the past three decades, there are cosiderable works i the literature devoted to developig statistical procedures that are resistat to outliers ad stable (or robust) with respect to deviatios from a give distributioal model. I particular, methods for robust regressio, estimatio, ad testig o regressio models have received much attetio. Amog these, procedures based o M-estimators play a importat ad complemetary role. Referece may be made to papers by Huber (1964, 1967, 1973, 1981), Bickel (1975), Yohai ad Maroa (1979), Heiler ad Willers (1988), Basawa ad Koul (1988), Bai, Rao ad Wu (1992) ad Bai, Rao ad Wu (1997) amog others. I liear models, the regressors are assumed to be liear fuctios of the regressio coefficiets. This assumptio may be due to mathematical coveiece i obtaiig the estimates by the traditioal Least Squares (LS) method. For relaxig this probably restrictive assumptio ad for seekig Received Jauary 3, 1996; revised May 13, AMS 1980 subject classificatio 62J05; 62H12. Key words ad phrases Geeral M-estimatio, asymptotics, multivariate liear model X Copyright 1997 by Academic Press All rights of reproductio i ay form reserved.

2 120 BAI AND WU robustess, Huber proposed the well kow M-estimatio. I M-estimatio, for a prechose dispersio fuctio \, we are cosiderig loss fuctios \(y i &X i ;). I this case, the mathematical advatage of liear regressors disappears, but the difficulty is overcome by the moder computig techiques. Therefore, comparig efforts i computig the M-estimators, the same amout of effort will be eeded for a more geeral model of oliear regressio, i.e., to cosider the target fuctios \(y i & g(x i, ;)), as that for liear models. I usual regressio models (liear or oliear), the distributios of the errors are assumed to be idepedet of the regressio coefficiets ;. However, i may practical situatios, the distributios of the errors may deped o the regressors. For example, the distributio of the observatio y i is log-ormally distributed with a mea value x$ i ;. The, the distributio of the error = i = y i &x$ i ; will deped upo x$ i ;. Because we oly eed to cosider the fuctio \(y i & g(x i, ;)) where the form of g(x i, ;) is ot essetial, we may simply cosider a more geeral form of \ i (y i, ;). Therefore, i this paper we shall itroduce a more geeral set-up for M-estimatio, which will cover all the above metioed models as its special cases. Let [y 1,..., y,...] be a sequece of radom vectors ad for each ; # 0, [\ 1 (y 1, ;),..., \ (y, ;),...] be a sequece of fuctios which are differetiable about ; for almost all y's, where 0 is a ope covex subset of R p kow as the parameter space. The the geeral M-estimate ; is defied as ay value of ; miimizig \ i (y i, ;). (1.1) Let i (y i, ;) deote the derivative of \ i (y i, ;) about ; if the derivative exists ad 0 otherwise. The, ; satisfies if the left side of (1.2) is cotiuous at ;,or } i (y i, ; ) }=mi ; } i (y i, ; )=0, (1.2) i (y i, ;) (1.3) }, otherwise. To ivestigate the asymptotics of the estimator, we eed to defie the ``true parameter.'' Assume that there exists a vector ; 0 # 0 ad for each i there are a oegative defiite matrix G i ad a fuctio ' i (;) such that E i (y i, ;)=G i (;&; 0 )+' i (;) (1.4)

3 GENERAL M-ESTIMATION 121 with ' i (;)=o(q i (;&; 0 )) as ;; 0, where Q i is some oegative defiite matrix. Although ; 0 may ot be uiquely determied by a sigle Eq. (1.4) for a fixed i, i practice, oe may show that ; 0 ca be uiquely determied by all equatios of type (1.4) for, 2,..., ( 0 ), uder certai reasoable coditios. Comparig the setup described above with that i a fudametal work of Huber (1967), oe fids that our model geeralizes Huber's, except that Huber defies his estimator T(x) as a approximate miimizer of his target fuctio istead of as the exact oe, (see his formula (1)). This assumptio is somewhat more realistic, sice the recursive computatio ca oly get a approximatio of the exact M-estimator, however, there are o ay theoretical differece i their asymptotic theory. I Sectio 2, we shall give some further assumptios ad state ad establish the asymptotic results of the geeral M-estimatio uder covex discrepacy fuctios. I Sectio 3, the mai results of Sectio 2 will be geeralized to the geeral M-estimatio uder geeral discrepacy fuctios. Several examples ad some discussios will be give i Sectio ASYMPTOTICS FOR CONVEX REGRESSION I this sectio, we assume that \ i (y i, ;) is a covex fuctio whe y i is fixed,, 2,... ad deote by i a measurable selectio of subgradiets of \ i. We also assume that oly o a set of probability zero i may have discotiuities. Let D i (y i, ;)=\ i (y i,;+; 0 )&\ i (y i,; 0 ), 2 i (y i, ;)= i (y i,;+; 0 )& i (y i, ; 0 ) ad 2= 2 i, Q ()= Q i, G()= G i, ad S(;, A)=;$A;. For coveiece of otatio, we shall write 2 i, Q ad G for 2 i (y i, ;), Q () ad G(), respectively, whe there is o cofusio. We make the followig assumptios. (A1) Let Q=Q() be a positive defiite matrix ad suppose that 0 a 1 <if * mi (G Q )sup * max (G Q )<a 2 <, ad that 0a 1 <if * mi (Q Q &1 ) sup * max (Q Q &1 )<a 2 <, where, ad i what follows, G Q =Q &12 GQ &12, * mi (A) ad * max (A) deote the smallest ad largest eigevalues of A, respectively. Q &1 0. Remark. I geeral, Q ca be chose as Q=E( i(y i, ; 0 )) ( i (y i, ; 0 ))$ ad Q i =E i (y i, ; 0 ) $ i (y i, ; 0 ). The Q =Q whe y 1, y 2,... are idepedet.

4 122 BAI AND WU (A2) Cov(D i(y i, ;), D j(y j, ;))=& i, j (;) ;$(Q i +Q j ) ; ad max i j=1 & i, j (;) 0, as ; 0, where D i(y i, ;)=D i (y i, ;)&;$ i (y i, ; 0 )&E(D i (y i, ;)&;$ i (y i, ; 0 )). Remark. We poit out that the secod part of Coditio (A2) is a cosequece if E 2 i 2$ i & i (;) Q i with max i & i (;) 0 (as ; 0) ad the sequece [y i ] is,-mixig with, 12 <. I fact, by Lemma 1 of Sectio 20 i Billigsley (1968), we have Cov(D i(y i, ;), D j(y i, ;)) 2, 12 i&j [Var(D i(y i, ;)) Var(D j(y j, ;))] 12. The by the covexity of \ i, we have Hece, D i (y i, ;)&;$ i (y i, ; 0 ) ;$2 i. Var(D i(y i, ;));$E2 i 2$ i ;. The, Coditio (A2) is true by choosig & ij 2 -, i&j & i & j. (A3) Q() &12 i(y i, ; 0 ) w D N(0, I p ). Remark. If y i 's are idepedet, the i (y i, ; 0 )'s are idepedet. Hece Assumptio (A3) is true uder Lideberg's coditio. I may applicatios of the mai theorems, the idepedece coditio holds. We have the followig theorems. Theorem >0, Uder the assumptios (A1) ad (A2), for ay fixed sup Q ; <+} 12 [D i (y i, ;)&;$ i (y i, ; 0 )]& 1 S(;, G) 2 0 i probability. } (2.1) The followig lemma is eeded i the proof of Theorem 2.1. Lemma 2.1. We have ED i (y i, ;)= 1S(;, G 2 i)+o(s(;, Q i )). (2.2) Proof. The lemma ca be proved followig the same procedure as the proof of Lemma 1 i Bai, Rao ad Wu (1992).

5 GENERAL M-ESTIMATION 123 Proof of Theorem 2.1. I order to prove (2.1), we oly eed to show that for ay subsequece [$] of positive itegers, there exists a subsequece ["] of the subsequece [$] such that " sup [D Q ; <+} i (y i, ;)&;$ i (y i, ; 0 )] 12 & 1 S(;, G) 2 0 a.s. as ". (2.3) } Let us make the trasformatio #=Q 12 ;. The, the assertio (2.1) becomes " sup # <+} By (A2) ad (A3), we have Var \ Therefore, [D i (y i, Q &12 #)&#$Q &12 i (y i, ; 0 )] & 1 2 S(Q &12 #, G) 0 a.s. as ". (2.4) } (D i (y i, Q &12 #)&#$Q &12 i (y i, ; 0 )) + = Cov(D i(y i, ;), D j(y j, ;)) i, j 2 By Lemma 2.1, ;$Q i ; j=1 & ij (;) 2#$Q &12 QQ &12 # max 1i 2a &1 1 # 2 max 1i j=1 j=1 & ij (Q &12 #) & ij (Q &12 #) 0, sice Q &1 0. [D i (y i, Q &12 #)&#$Q &12 i (y i, ; 0 ) &ED i (y i, Q &12 #)]0 i probability. ED i (y i, Q &12 #)= 1 2 S(Q&12 #, G)+o(S(Q &12 #, G)) = 1 2 S(#, GQ )+o(1),

6 124 BAI AND WU where G Q is defied i Assumptio (A1). By (A1), there is a subsequece [$$$] of [$] such that G Q ($$$) G Q 0 >0 as ij, where G Q 0 is some positive defiite matrix of costats. The, for all fixed #, we have $$$ [D i (y i, Q &12 #)&#$Q &12 i (y i, ; 0 )] & 1 S(#, 2 GQ 0 )0 i probability as $$$. Usig diagoalizatio techique, we ca choose a subsequece ["] of [$$$] such that " [D i (y i, Q &12 #)&#$Q &12 i (y i, ; 0 )] 1 2 S(#, G Q 0 )0, a.s. as ", for each # i ay give coutable dese set of R p. Sice " [D i(y i, Q &12 #) &#$Q &12 i (y i, ; 0 )] is covex i # ad 1 S(#, 2 GQ 0 ) is a cotiuously differetiable covex fuctio i #, we obtai, by the geeralized Theorem 10.8 of Rockafellar (1970), " sup Q ; <+} 12 [D i (y i, ;)&; i (y i, ; 0 )]& 1 S(#, 2 GQ) 0 0 a.s. } Therefore, (2.4) follows ad the proof of Theorem 2.1 is complete. Theorem 2.2. I additio to the assumptios of Theorem 2.1, we assume the costat a 1 >0 i the coditio (A1), the ; ; 0 i probability. (2.5) Theorem >0, Uder the assumptios of Theorem 2.2, we have, for ay sup Q &12 (2&G;) 0 i probability. (2.6) Q 12 ; <+ Based o Theorem 2.1, Theorems 2.2 ad 2.3 ca be proved by the same approach of Bai, Rao ad Wu (1992).

7 GENERAL M-ESTIMATION 125 Theorem 2.4. Uder the assumptios of Theorem 2.2, we have ; =; +o p (&Q &12 &), (2.7) where ; =; 0 +G &1 Cosequetly, if we further assume (A3) is true, the i (y i, ; 0 ). (2.8) Q &12 G(; &; 0 ) N(0, I). (2.9) Proof. I order to prove (2.7), we eed oly to show that The (2.9) follows whe (A3) is true. By (A1), we have Q 12 (; &; ) 0 i probability. (2.10) Q 12 (; &; 0 )=O p (1). Hece, by (2.1) ad the defiitio of ;, it follows that D i (y i, ; &; 0 )& 1 2 S(; &; 0, G)0 i probability. (2.11) Take {>0. For a sequece [+ ] with +, by (2.1), we have sup Q 12 ; + +{} [D i (y i, ;)&;$ i (y i, ; 0 )]& 1 S(;, G) 2 0 i probability, } which, together with (2.11), implies that sup Q 12 (;&; ) ={} [\ i (y i, ;)&\ i (y i, ; ) & 1 S(;&;, G)) 2 0 i probability. (2.12) } Sice for Q 12 (;&; ) ={, S(;&;, G)* mi (G Q ) { 2,

8 126 BAI AND WU by (2.12), we get P( Q 12 (; &; ) {)0. The, (2.10) ad hece the theorem is proved. Now, cosider a test of the hypothesis H 0 H$;=! 0, where H is a p_q matrix of rak q. Let ; deote the solutio of mi \ i (y i, ;) H$;=! 0 ad ; be defied as before. The we have the followig theorem. Theorem 2.5. Suppose the assumptio (A3) holds i additio to the assumptios of Theorem 2.2. The, uder the ull hypothesis, } [\ i (y i, ; )&\ i (y i, ; )]& 2} 1 K$ i (y i, ; 0 ) } 2 0 i probability, } (2.13) ad where (H$; &! 0 )$ (H$G &1 QG &1 H) &1 (H$; &! 0 ) / 2 q, (2.14) K=G &1 H(H$G &1 H) &12, is a p_q matrix ad / 2 q deotes a chi-square radom variable with q degrees of freedom. Proof. By (2.1), we get By (2.8) ad (2.10), it follows that Q 12 (; &; 0 )=Q 12 G &1 [\ i (y i, ; )&\ i (y i, ; 0 )]=& 2} 1 G&12 i (y i, ; 0 )+o p (1). (2.15) i (y i, ; 0 ) } 2 +o p (1). (2.16) Let W be a p_( p&q) matrix such that K$W=0 ad W$W=I p&q (with two extreme cases where we defie K=0, W=I p if q=0 ad K=I p, W=0 if p=q). The H$(;&; 0 )=0 ;=; 0 +W.

9 GENERAL M-ESTIMATION 127 That meas, uder ull hypothesis, we may assume ;=; 0 +W. Let ^ be the M-estimator of with respect to the dispersio fuctios \ i (y i, ; 0 +W). Note that the gradiet of \ i (y i, ; 0 +W) with respect to is W$ i (y i, ; 0 +W) with EW$ i (y i, ; 0 +W)=W$G i W(& 0 )+o(w$q i W(& 0 )). Let ; =; 0 +W^. Similar to (2.16), we obtai [\ i (y i, ; )&\ i (y i, ; 0 )]=& 1 2} (W$GW)&12 W$ uder the ull hypothesis. Hece, [\ i (y i, ; )&\ i (y i, ; 0 )]= 2} 1 K$ by oticig that i (y i, ; 0 ) } i (y i, ; 0 ) } 2 KK$=G &1 &W(W$GW) &1 W$ =G &1 H(H$G &1 H) &1 H$G &1. By Theorem 2.4 ad (2.15), it follows that (; &; 0 )$ H(H$G &1 QG &1 H) &1 H$(; &; 0 ) / 2 q. The proof of the theorem is fiished. 2 +o p (1), +o p (1), (2.17) 3. COMMENTS ON NON-CONVEX DISPERSIONS I may situatios of robust estimatio, the dispersio fuctios may ot be covex. However, the results i Sectio 2 ca be easily exteded to the case that each dispersio fuctio is a differece of two covex fuctios, which covers almost all useful cases of M-estimatio by the followig fact. Theorem 3.1 (See Theorem 4.2 of Bai, Rao ad Wu (1997)). Every fuctio which is cotiuously twice differetiable ca be writte as a differece of two covex fuctios which are strictly covex. Therefore, we shall devote this sectio to some geeral commets o M-estimatio defied by dispersio fuctios each of which is a differece

10 128 BAI AND WU of two covex fuctios. We shall oly state the mai results without detailed mathematics. To begi with, we eed to clarify the followig cocept of the M- estimators. Whe \ i is ot covex, the global miimizer of (1.1) may ot exist or may ot be cosistet eve whe it exists. See Bai, Rao ad Wu (1997) for examples. However, it ca be show that the miimizatio problem (1.1) must have at least oe local miimizer aroud the true value of the regressio parameter eve whe the global miimizer does ot exist. At the same time, there arises aother problem that (1.1) may have may local miimizers i such case. Therefore, we have to clarify which miimizer satisfies the asymptotic properties discussed i this sectio. For this ed, let $ be a positive costat ad deote by ; ($) the absolute miimizer of (1.1) i the eighborhood [; Q 12 (;&; 0 ) $], where Q is defied i (B1). I case that the solutio is ot uique, ; ($) deotes either oe of the solutios. If there is a sequece [$ $ ] such that Q 12 (; ($ )&; 0 )=O(1), the we deote ; =; ($ ). It should be oted that the defiitio of ; is actually idepedet of the choice of the sequece [$ ] because of the fact that Q 12 (; ($ 2 )&; 0 ) $ 1 $ 2 implies that ; ($ 1 )=; ($ 2 ). I the preset sectio, we shall show that such a sequece always exists. Let \ i (y i, ;)=\ i1 (y i, ;)&\ i2 (y i, ;), where \ i1, \ i2 are strictly covex as fuctios of ; with fixed y i, ad have derivatives i1 (y i, ;) ad i2 (y i, ;) about ;, respectively, for, 2,... It is assumed that the uio of discotiuity set of i1, i2,, 2,..., has zero probability ad E ij (y i, ;)=G ij (;&; 0 )+' ij (;), with G ij 0 ad ' ij (;)=o(q i (;&; 0 )) as ;; 0, j=1, 2. As i Sectio 2, for each j=1 ad 2, defie D ij =D ij (y i, ;)=\ ij (y i, ;+; 0 )&\ ij (y i, ; 0 ), 2 ij =2 ij (y i, ;)= ij (y i, ;+; 0 )& ij (y i, ; 0 ), 2( j)= 2 ij, ad G j =G j ()= G ij. Ad deote i (y i, ;)= i1 (y i, ;)& i (y i2, ;), 2=2(1)&2(2) ad G=G()=G 1 &G 2. We make the followig assumptios. (B1) Suppose that G satisfies the Assumptio (A1) with a 1 >0 ad G 2 satisfies (A1) with a 1 0, where G=G 1 &G 2. (B2) The Assumptio (A2) is true for both \ i1 ad \ i2. (B3) Same as (A3). Theorem 3.2. Uder the assumptios (B1) ad (B2), for ay fixed +>0, sup Q ; <+} 12 [D i (y i, ;)&;$ i (y i, ; 0 )]& 1 S(;, G) 2 0 i probability. } (3.1)

11 GENERAL M-ESTIMATION 129 Proof. By Theorem 2.1, both D i1 associated with i1 (y i, ; 0 ) ad D i2 associated with i2 (y i, ; 0 ) satisfy (2.1). Thus, (3.1) follows. Theorem 3.3. Uder the assumptios of Theorem 3.2, we have a local miimizer ; such that ; ; 0 i probability. The proof of this theorem is completely the same as that of Theorem 2.2. Here, we would like to remid the reader that the local absolute miimizer is isolated i a small ball with ceter ;, where ; is similarly defied as i (2.8). Cocretely speakig, we have the followig Remark. Sice (3.1) is true for all +>0, there exists a sequece of + such that (3.1) is still true whe + is replaced by +. Usig similar argumets as i the proof of Theorem 2.2, oe ca prove that with a probability tedig to oe, there is o local miimizer i the hyper rig 0<$< ;&; <+. Theorem >0, Uder the assumptios of Theorem 3.2, we have, for ay sup Q &12 (2&G(;&; 0 )) 0 i probability. Q 12 (;&; 0 ) <+ This theorem follows from the fact that the above covergece is true for both 2(1) ad 2(2). Theorem 3.5. Uder the assumptios (B1), (B2) ad (B3), we have Q &12 G(; &; 0 ) N(0, I). For testig the hypothesis H 0 H$;=! 0, where H is a p_q matrix of rak q. Let ; deote the solutio of mi H$;=! 0 \ i (y i, ;) ad ; be defied as before. The, uder the assumptios (B1), (B2) ad (B3), Theorem 2.5 is still true whe the dispersio fuctios \ i 's are replaced by differeces of covex fuctios. Sice its mathematical expressios are exactly the same as those i Theorem 2.5, we do ot write it out as a theorem.

12 130 BAI AND WU 4. SOME EXAMPLES I this sectio, we implicitly assume that \ or \ i are covex fuctios or differeces of covex fuctios without statig. For most well kow results i the literature, Assumptios (A13) or (B13) are trivially satisfied without verificatio, which is eve ot clearly stated here. We oly give a bit more details to cases which to ot follow from the usual M-estimatio The usualm-estimatio i liear regressio models. Cosider a geeral multivariate regressio model y i =X$ i ;+e i,,..., (4.1) where e i,,...,, are vectors of radom errors, X i,,...,, are desig matrices ad ; is a p-vector of ukow parameters. I Bai, Rao ad Wu (1992) ad Bai, Rao ad Wu (1997), a geeral asymptotic theory of M-estimatio is developed uder dispersios formed by a covex fuctio ad a differece of covex fuctios, respectively. That meas, the M-estimator ; is defied by miimizig \(y i &X$ i ;) (4.2) for a suitable choice of the fuctio \, or solvig the equatios X i (y i &X$ i ;)=0 (4.3) for a suitable choice of the fuctio. A well kow example for the covex ad robust choice of is Huber's example. The followig are some examples of o-covex dispersios 1. (x)=2x(1+x 2 ). The, we ca choose 1 (x)=2x(1+x 2 ) for x 1 ad =sig(x) for x >1, whereas 2 (x)=0 for x 1 ad =sig(x)&2x(1+x 2 ) for x >1. 2. Hampel's, i.e., for costats, 0<a<b<c, (x)=x for x a, =a sig(x) for a< x b, =a sig(x)(c& x )(c&b) for b< x c ad =0 otherwise. I this case, we ca choose 1 (x)=x for x a ad =a sig(x) for x >a whereas 2 (x)=0 for x b, =a sig(x) ( x &b)(c&b) for b< x c ad =a sig(x), otherwise. For these choices, the G i ad Q i are defied by the expectatio of ( y i &x$ i ;) ad the covariace of ( y i &x$ i ; 0 ), for,...,. Uder very geeral ad mild regularity coditios, the assumptios (A1)(A3) or

13 GENERAL M-ESTIMATION 131 (B1)(B3) are satisfied. For details, see Bai, Rao ad Wu (1992) ad Bai, Rao ad Wu (1997) The usual M-estimatio i oliear regressio models. Cosider a geeral multivariate oliear regressio model y i =f i (;)+e i,,..., (4.4) where e i,,...,, are vectors of radom errors, f i,,...,, are give fuctios ad ; is a p-vector of ukow parameters. As a special case, f i (;)=f(x i, ;) with give f ad desig matrices X i,,...,. I M-estimatio of ;, oe takes \ i (y i, ;)=\(y i &f i (;)) ad i (y i, ;)= &(df td;) (y i i&f i (;)). Uder the coditios that E((e+c))=Ac+o(c) (or =Ac+O( c 2 )) with A>0 ad E((e) (e) t )=B>0 ad some mior coditios, say, the derivatives df i d; are equi-cotiuous at ; 0, we have G i = (df td;)a(df i id;) ;=;0 ad Q i =(df td;)b(df i id;) ;=;0. Furthermore, if some regularity coditios o the growth rate ad osigularity coditios o (df i d;) ;=;0 are met, say max 1i df t i d;\ j=1 df j d; df t &1 j df i 0, d;+ d; } ;=; 0 oe ca verify that the coditios of Sectios 2 or 3 are satisfied The geeral M-estimatio i liear models. I liear models, the geeral M-estimatio ; of ; is defied by a value which miimizes or a solutio of the equatio w i (X i ) \(y i &X i ;) w i (X i ) X i (y i &X i ;)=0, where [w i (X i )] is a set of weights depedig o the desig. The, this is equivalet to select \ i (y i, ;)=w i (X i ) \(y i &X i ;) ad i (y i, ;)=&w i (X i ) X i (y i &X i ;). This is the well kow Mallow estimatio. For geeral results ad the choices of [w i (X i )], see Hampel et al. (1986).

14 132 BAI AND WU 4.4. The M-estimatio i AR model. Cosider the AR time series x =; 1 x &1 +}}}+; p x &p +=, where ;=(; 1,..., ; p )$ is a vector of ukow autoregressive coefficiets ad [= 1, = 2,...] is a sequece of iid. radom errors. The, M-estimator ; of ; is defied by choosig \ i (y i, ;)=\(x i &; 1 x i&1 &}}}&; p x i&p ), where \ is a suitably chose dispersio fuctio ad y i =(x i,..., x i& p )$ for p<i. If is the derivative of \ satisfyig E(=+a)=*a+o(a) ad E 2 (=)=_ 2, the we have i (y i, ;)=&x i (= i &x$ i (;&; 0 )), where x i =(x i&1,..., x i& p )$. Let 7=E(x i x$ i ). The, oe ca verify that ad E( i (y i, ;))=*7(;&; 0 )+o( ;&; 0 ) (4.5) E( i (y i, ; 0 ) i (y i, ; 0 )$)=_ 2 7. (4.6) Sice a AR time series is,-mixig whose, fuctio is expoetially decayig, by (4.5), (4.6) ad the remark below Assumptio (A2), Assumptio (A2) is true. Hece, the coditios of Sectio 2 are true ad cosequetly - (; &; 0 ) w D N(0, A), where A=* &2 _ 2 7 &1. This has bee studied by R. D. Marti (1980, 1981), to which the readers are referred for more details The MLE of regressio coefficiets of meas of expoetial variables. Let y i be expoetially distributed with a mea x$ i ;, where x i is kow. We may write this model as y i =x$ i ;+= i, where = i has a zero mea ad variace 2(x$ i ;) 2. That meas, the error distributios deped o the ukow parameters. Let y 1,..., y be idepedet. The MLE ; of ; is to miimize \ y i x$ i ; +log(x$ i;) +, uder the restrictio that mi i x$ i ;>0.

15 GENERAL M-ESTIMATION 133 Correspodig to our model, we have \ i (y i, ;)=(y i x$ i ;)+log(x$ i ;), ad the i (y i, ;)=&(x i (y i &x$ i ;)(x$ i ;) 2 ). By elemetary calculatio, we obtai ad E i (y i, ;)= x ix$ i (;&; 0 ) (x$ i ;) 2 E i (y i, ; 0 ) $ i (y i, ; 0 )=2x i x$ i. The, it is easy to verify that the assumptios (A13) hold uder the coditios ad max i _ x$ i\ if &1 x i x$ i+ mi x$ i ; 0 x i $>0 i x i\ max \ 1, 1 x$ i ; 0++& 0, as. Similar cosideratio may apply to other parametric models, such as iverse Gaussia regressio (see Chhikara ad Folks (1989)) The oliear regressio i EIVR models. There is much work i Error I Variables Regressio models. For a applicatio to this model, we refer to Gleser (1990) ad Stefaski (1985). The EIVR (structural) model is give by y=h(!, ;)+=, x=!+$, where!, $ ad = are idepedet radom vectors of dimesios s, s ad t, respectively ad ; is ukow vector of dimesio p. Suppose that (y i, x i ),, 2,...,, are idepedet observatios o this model. As Gleser poited out, a aive approach which simply substitutes! by x usually leads to icosistet estimators of ;. Now let x be a fuctio of x which will be determied later. We cosider the M-estimator of ; by miimizig \(y i &h(x i, ;)).

16 134 BAI AND WU I this case, we have i (y i, ;)=&H(x i, ;) (y i &h(x i, ;)) with H(x, ;)=(;) h$(x, ;). Uder the coditio o the \ fuctio made i Sectio 1, we should have E i (y i, ;)=EH(x i, ;)[G(h(x i, ;)&h(! i, ; 0 ))+o(h(x i, ;)&h(! i, ; 0 ))]. Thus, if we ca choose x such that E(h(!, ; 0 ) x)=h(x, ; 0 ), (4.7) the, uder some mior cotiuity coditios o H ad h, we shall have ad where E i (y i, ;)=E(H(x, ; 0 ) GH$(x, ; 0 ))(;&; 0 )+o(;&; 0 ) E i (y i, ; 0 ) $ i (y i, ; 0 )=7, 7=E(H(x, ; 0 ) (=+h(!, ; 0 )&h(x, ; 0 )) $(=+h(!, ; 0 ) &h(x, ; 0 )) H$(x, ; 0 )). Cosequetly, the coditios of the mai theorems are satisfied ad hece the mai theorems hold true. Remark. Whe h(!, ;) is a liear fuctio i both! ad ; ad the uderlyig distributios are multivariate ormal, the coditio (4.7) is x =E(! x)=4x+(i&4) +. which is idepedet of the parameter ; to be estimated. I case that 4 ad + are ukow, they ca be estimated from the x observatios. The, these quatities ca be replaced by their cosistet estimates. We shall ot give further details i this remark. We oly wat to remid the reader that the estimator or ; defied here are cosistet to the true value of the iterestig parameter, ulike those defied by Gleser which would coverge to somethig else (see his Theorem 3.1). A key reaso to guaratee our approach is applicable is that the fuctio x (see (4.7)) should be idepedet of ; 0, that is, the fuctio x =x (x) is either completely kow, or ivolves with parameters which is estimable by oly the x-sequece. For the least squares approach i liear case, this is possible.

17 GENERAL M-ESTIMATION 135 ACKNOWLEDGMENTS The authors thak the referee for his helpful commets. Y. Wu's research was partially supported by the Natural Scieces ad Egieerig Research Coucil of Caada. REFERENCES [1] Bai, Z., Rao, C. R., ad Wu, Y. (1992). M-estimatio of multivariate liear regressio parameters uder a covex discrepacy fuctio. Statistica Siica [2] Bai, Z., Rao, C. R., ad Wu, Y. (1997). M-estimatio of multivariate liear regressio by miimizig the differece of two covex fuctios. I Hadbook of Statistics, Vol. 15, to appear. [3] Basawa, L. V., ad Koul, H. L. (1988). Large-sample statistics based o quadratic dispersio. Iterat. Statist. Rev [4] Bickel, P. J. (1975). Oe-step Huber estimates i the liear model. J. Amer. Statist. Assoc [5] Billigsley, P. (1968). Covergece of Probability Measures. Wiley, New York. [6] Chhikara, R. S., ad Folks, J. L. (1989). The Iverse Gaussia Distributio; Theory, Methodology ad Applicatios. Dekker, New York. [7] Gleser, L. J. (1990). Improvemets of the Naive approach to estimatio i oliear Error-i-variables regressio models. Cotemporary Mathematics [8] Hampel, F. R., Rochetti, E. M., Rousseeuw, P. J., ad Stahel, W. A. (1986). Robust StatisticsThe Approach Based o Ifluece Fuctios. Wiley, New York. [9] Heiler, S., ad Willers, R. (1988). Asymptotic ormality of R-estimates i the liear model. Statistics [10] Huber, P. J. (1964). Robust estimatio of a locatio parameter. A. Math. Statist [11] Huber, P. J. (1967). The behavior of maximum likelihood estimates uder o-stadard coditios. I Proceedigs of the Fifth Berkeley Symposium o Mathematical Statistics ad Probability, Vol. 1, Uiv. Calif. Press, pp [12] Huber, P. J. (1973). Robust regressio. A. Statist [13] Huber, P. J. (1981). Robust Statistics. Wiley, New York. [14] Marti, R. D. (1980). Robust estimatio i autoregressive models. I Directios i Time Series Vol. (D. R. Brilliger ad G. C. Tiao, Eds.), pp IMS Publicatio, Haywood CA. [15] Marti, R. D. (1981). Robust methods for time series. I Applied Time Series II (D. F. Fidley, Ed.), pp Academic Press, New York. [16] Stefaski, L. A. (1985). The effects of measuremet error o parameter estimatio. Biometrika [17] Yohai, V. J., ad Maroa, R. A. (1979). Asymptotic behavior of M-estimators for the liear model. A. Statist