MU-Estimation and Smoothing

Transkript

1 Joural of Multivariate Aalysis 76, (2001) doi: jmva , available olie at o MU-Estimatio Smoothig Z. J. Liu Mississippi State Uiversity C. R. Rao Pe State Uiversity Received February 22, 1999; revised Jauary 10, 2000 I the M-estimatio theory developed by Huber (1964, A. Math. Statist. 43, ), the parameter uder estimatio is the value of % which miimizes the expectatio of what is called a discrepacy measure (DM) $(X, %) which is a fuctio of % the uderlyig rom variable X. Such a settig does ot cover the estimatio of parameters such as the multivariate media defied by Oja (1983) Liu (1990), as the value of % which miimizes the expectatio of a DM of the type $(X 1,..., X m, %) where X 1,..., X m are idepedet copies of the uderlyig rom variable X. Arcoes et al. (1994, A. Statist. 22, ) studied the estimatio of such parameters. We call such a M-type MU-estimatio (or +-estimatio for coveiece). Whe a DM is ot a differetiable fuctio of %, some complexities arise i studyig the properties of estimators as well as i their computatio. I such a case, we itroduce a ew method of smoothig the DM with a kerel fuctio usig it i estimatio. It is see that smoothig allows us to develop a elegat approach to the study of asymptotic properties possibly apply the NewtoRaphso procedure i the computatio of estimators Academic Press AMS 1991 subject classificatios: 62F35, 62F12, 62H12. Key words phrases: data depth, discrepacy measure, estimatig equatio, kerel, multivariate media, M-estimatio, MU-estimatio, U-statistic. 1. INTRODUCTION Kerel smoothig has played a importat role i oparametric desity estimatio oparametric regressio. The empirical distributio fuctio of a i.i.d. sample is the best estimator of the uderlyig distributio fuctio. However, a desity estimator caot be derived from a empirical distributio fuctio by differetiatig because it is ot differetiable. The idea behid kerel smoothig is that the covolutio of a X Copyright 2001 by Academic Press All rights of reproductio i ay form reserved.

2 278 LIU AND RAO empirical distributio fuctio a differetiable fuctio is a differetiable fuctio; the oe may differetiate the covolutio to have a desity estimator. The smoothess of the covolutio its deviatio from the uderlyig distributio is cotrolled by a smooth parameter called bwidth. Such a procedure may itroduce bias. However, i this way oe may avoid some difficulties i solvig some statistical problems. Pricipal compoet regressio ridge regressio are other examples of sacrificig ubiasedess. Several smoothig methods have bee itroduced i the least absolute deviatios (LAD) regressio. Horowitz (1992, 1998) used smoothig similar to the kerel method for a biary respose model. Liu Jag (1998) used kerel smoothig for LAD estimators. These methods were itroduced to overcome difficulties i theoretical studies computatios. Recetly, Liu (1999) foud a coectio betwee the LAD the M-estimators. He showed that every M-estimator with a positive breakdow poit a differetiable discrepacy fuctio is a smoothed LAD estimator. I this paper we study kerel smoothig of M-estimators uder a geeral settig called MU (or +)-estimators. We cocetrate o asymptotic properties of the kerel smoothed +-estimators. Possible beefits of smoothed +-estimatio i computatios have bee observed i simulatio results. Detailed results of simulatio studies will be reported i a future paper. Let us cosider a parameter % F # R q of a p-variate distributio F where % F =arg mi[2(+, F)], (1) 2(+, F)=E F $(X 1,..., X m, +). (2) E F ( } ) deotes the expectatio with respect to F, $(x 1,..., x m, %): R mp _ R q R is a symmetric fuctio of x 1,..., x m for every fixed %, called the discrepacy measure (DM), X 1,..., X m are idepedet copies of a rom variable X distributed as F. Let X 1,..., X be a sample of idepedet observatios o XtF, defie F as the empirical distributio fuctio based o X 1,..., X. The +-estimate of %=(% 1,..., % q ) is defied as where we defie 2(%, F )= \ m+ &1 % =arg mi[2(%, F )], (3) : 1i 1 <i 2 }}}<i m $(X i1,..., X im, %). (4)

3 +-ESTIMATION 279 For every fixed %, 2(%, F ) is a U-statistics. We call (3) a +-estimate because it is a M-type estimate based o a U-statistics-type DM. If (4) is differetiable, the %, a estimator of %, is a solutio of where D is the differetial operator D2(%, F )=0, (5) D= \, D 2 = \ 2 % i+1_q % i % j+. (6) q_q If the secod derivative exists, we ca use a iterative procedure such as NewtoRaphso to compute the solutio of (5). Further the asymptotic properties of the estimator ca be studied without heavy assumptios. If the DM is ot differetiable, as i the case of the media estimator of Oja (1983) Liu (1999), the the problem of computig the +-estimate derivig its asymptotic properties eeds complex approaches. To overcome the possible difficulties, we propose to smooth the DM usig a differetiable kerel fuctio, which results i a differetiable fuctio. We work with such a smoothed DM derive the asymptotic properties of the resultig +-estimators. Let k(u): R q R be a differetiable fuctio, called a kerel fuctio, such that R q k(u) du=1. (7) Usig such a k(u), we obtai the smoothed versio of the DM by covolutio $ (x 1,..., x m, %)= $(x 1,..., x m, %&h u) h(u) du, (8) R q where h is a bwidth such that [h, =1, 2,...] decreases to zero as teds to ifiity. The derivative of $ is D$ (x 1,..., x m, %)= 1 $(x h q 1,..., x m, u) Dk R q \%&u h + du. (9) Let 2 (%, F )= \ m+ &1 : 1i 1 <i 2 }}}<i m $ (X i1,..., X im, %). (10)

4 280 LIU AND RAO The the +-estimator obtaied from the smoothed DM is a solutio of the equatio D2 (%, F )=0. (11) Sice we have the estimatig equatio i a explicit form, we may modify stard computatioal techiques to obtai the estimator %. The mai issue i computatio will be bwidth selectio to esure that o derivative vaishes i the iterative rage of the NewtoRaphso method the covergece takes place with low bias. Choosig large bwidth will guaratee covergece of the NewtoRaphso method. Our prelimiary study showed that we ca use a large bwidth reduce the bias at the same time. The detailed study will be reported i a future paper. We ow ivestigate the asymptotic stochastic properties of the smoothed +-estimator. We establish the cosistecy, asymptotic ormality, asymptotic represetatio of % i Sectio 3 of the paper. A discussio of the smoothed +-estimator of the media whe p=1 m=1 the multivariate media of Liu (1990) is give i Sectio 4, the results are compared with those obtaied by Arcoes et al. (1994) usig the usmoothed DM. For coveiece easy uderstig of the paper, we list some symbols statemets i the followig: v [X i, i=1,..., ] is a i.i.d. sample from a p-variate distributio F. f(x) deotes the desity fuctio F the empirical distributio. v &}& deotes the Euclidea orm. v C deotes a positive umber which may take differet values i differet places. Deote a = &1 log log. A matrix may be deoted as O(}) (o( } )) if its orm is O(}) (o(})). E( } ) cov( } ) deote mea variacecovariace matrix, respectively. v Coditios for chagig the order betwee itegrals, betwee derivatives, betwee itegral derivative are assumed to be satisfied. v k(u): R q R is a fuctio, called a kerel fuctio, such that R q k(u) du=1. [h, =1, 2,...] is a sequece of positive umbers called the bwidth. v $(x, %)=$(x 1,..., x m, %): R mp _R q R is a fuctio called a DM which is symmetric with respect to permutatios of (x 1,..., x m ). A kerel smoothed DM is $ (x, %)= R q $(x 1,..., x m, %&h u) k(u) du.

5 +-ESTIMATION 281 v Deote by A i, j the elemet i the ith row jth colum of matrix A. A$ deotes the traspose of matrix A D= \, D 2 =DD$, D 3 =D (DD$), % i+1_q D i = % i, D 2 i, j = 2 % i % j, D 3 i, j, l = 3 % i % j % l, k 1 (u)=dk(u), k 2 =D 2 k(u), 2(%, F)=E F $(X 1,..., X m, %), 2(%, F )= \ m+ &1 : 1i 1 <i 2 }}}<i m 2 (%, F)=E F $ (X 1,..., X m, %), 2 (%, F )= \ m+ &1 : 1i 1 <i 2 }}}<i m $(X i1,..., X im, %), $ (X i1,..., X im, %). G(%, F)=D2(%, F), G(%, F )=D2(%, F ), G (%, F)=D2 (%, F), G (%, F )=D2 (%, F ). S(%, F)=D 2 2(%, F), S(0, F)=D 2 2(%, F) %=0. 7=cov(E(D$(X 1,..., X m,0) X 1 )), 7 =cov(e(d$ (X 1,..., X m,0) X 1 )) 2. EXISTENCE AND CONSISTENCY Uder some regularity coditios, the strog law of large umbers of the U-statistic implies that G (%, F ) will coverge to G(%, F) for every fixed %, therefore whe G(%, F)=0 is solvable, the existece of a solutio of (11) depeds o the uiform closeess of G (%, F ) G(%, F). Theorem 1. Assume: A1. 2(%, F) has a uique miimum value, attaied at %=% 0, which may be take as zero without loss of geerality. G(%, F) S(%, F) exist for % # B, a ope eighborhood of 0, S(%, F) is cotiuous positive defiite for every % # B.

6 282 LIU AND RAO A2. The kerel fuctio chose has a compact support is three times differetiable. D 2 $ (x 1,..., x, %) is cotiuous has full rak for every % # B. A3. The bwidth sequece is such that A4. lim h =0. (12) lim sup &G (%, F )&G (%, F)&=0, w.p.1. (13) % # B The for large, (11) has a solutio %, w.p.1 lim % =0, w.p.1. (14) Remark 1. It is almost impossible to prove A4 without massive assumptios. A sufficiet coditio for A4 is that $(x 1,..., x m, %) ca be uiformly approximated by a step fuctio, i which case a proof similar to that of Theorem 5 i this paper ca be used. It is much easier to verify A4 for special cases. Differet sets of coditios may require differet approaches to prove A4. For some results related to the verificatio of A4, referece may be made to Pollard (1984). Remark 2. Whe $, the DM, is twice differetiable, o smoothig is eeded. I such a case the bwidth ca be chose to be 0. Proof. We prove Theorem 1 for q=2 h {0. Similarly, oe ca prove the theorem for other cases. Note that G (%, F)= R p G(%&h u, F) k(u) du. (15) The by coditios A2, A3, A4, (15) we have lim sup &G (%, F)&G(%, F)&=0. (16) % # B By coditio A1 the implicit fuctio theorem, there exist cotiuous curves C 1 (% 1 )=(% 1, c 1 (% 1 )) C 2 (% 1 )=(% 1, c 2 (% 1 )) such that C i (t)/b, C i (0)=(0, 0), G (i) (C i (t), F)=0, i=1, 2, (17)

7 +-ESTIMATION 283 where (G (1), G (2) ) T =G. By coditio A4 (16), it is obvious that lim sup &G (%, F )&G(%, F)&=0, w.p.1. (18) % # B By coditio A2, (18), the implicit fuctio theorem, for every =>0 large, there exist cotiuous curves C,1 (% 1 )=(% 1, c,1 (% 1 )) C,2 (% 1 )=(% 1, c,2 (% 1 )) such that C, i (% 1 )/B, G (i) (C, i(% 1 ), F )=0, i=1, 2, (19) C i (% 1 )&=<C, i (% 1 )<C i (% 1 )+=, w.p.1, i=1, 2. (20) Coditio A1 implies that C 1 (% 1 ) C 2 (% 1 ) cross with each other at a uique poit which is (0, 0). Equatio (20) implies that C,1 (% 1 ) C,2 (% 1 ) cross with each other at a poit i B whe = is small eough. This cross poit is %.Let= 0, which requires. The % 0, w.p.1, which establishes the desired result. 3. ASYMPTOTIC NORMALITY AND ASYMPTOTIC REPRESENTATION We first cosider a +-estimator with a twice differetiable discrepacy fuctio. Sice we proved the existece cosistecy of +-estimators uder certai coditios, we assume % exists is cosistet i the followig. Theorem 2. Assume that $ the DM is twice differetiable with respect to % that % exists is strogly cosistet. Let a = &1 log log. Further assume: B1. Both D $(X 1,..., X m, %) D 2 $(X 1,..., X m, %) have secod order momets for every % # B. B2. G(0, F ) is ot a degeerate U-statistic; i.e., 7 is positive defiite. The % =&S &1 (0, F) G(0, F )+O(a ), w.p.1, (21) 12 S(0, F) % N(0, m 2 7), i dist., as. (22)

8 284 LIU AND RAO Proof. By Taylor's theorem coditio B1 G(0, F )=G(0, F )&G(%, F ) =&DG(0, F ) % +O(&% & 2 ), w.p.1. (23) By the law of iterated logarithm for U-statistics, we have Therefore G(0, F )=O(a 12 ), w.p.1, (24) DG(0, F )=S(0, F)+O(a 12 ), w.p.1. (25) % =&S &1 (0, F) G(0, F )+O(a 12 &% &)+O(&% & 2 ), w.p.1, (26) which proves (21). Sice G(0, F ) is a odegeerate U-statistic, by Hoeffdig's (1948) cetral limit theorem for U-statistics, (22) is true. The followig results relate to smoothed estimators, i.e., those obtaied by usig a smoothed DM. Theorem 3. hold Assume % exists is strogly cosistet the followig C1. There exists :>0 such that cov(d 2 $ (X 1,..., X m, 0))=O(h &: ) O cov(d 3 $ (X 1,..., X m, 0))=O(h &:&2 ). (27) C2. There is a iteger l>2 such that 2(%, F) is l times cotiuously differetiable, lim 7 =7 exists is positive defiite. C3. k(u) has a compact support, is twice differetiable, there is a iteger r1 such that m u l1 k(u) du=0, for l 1 }}}ulm m j0 0< : R q where u=(u 1,..., u m ). C4. Choose h =O(a 1(2r+2) ). The j=1 l j rl&2, (28) % =&S &1 (0, F) G (0, F )+O(a (4r+:)(4r+2:) ), w.p.1, (29)

9 +-ESTIMATION S(0, F)(% &E(D$ (X 1,..., X k,0)))n(0, k 2 7), i dist. as. (30) Remark 3. I coditio C1, the value of : depeds o the smoothess of the uderlyig desity the DM. If the DM is twice differetiable, the cov(d 2 $ (X 1,..., X m,0))=o(1). I fact :4. Remark 4. I the worst case where :=4, (29) becomes % =&S &1 (0, F) G (0, F )+o(a (r+1)(r+4) ), w.p.1. (31) The remaider ca be as close to O(a ) as possible if r is large eough. I the best case where the DM is twice differetiable (29) becomes % =&S &1 (0, F) G (0, F )+O(a ), w.p.1. (32) Remark 5. I fact EG (0, F )= R q D2(&h u, F) k(u) du=o(h r+1 ). (33) Coditio C4 implies lim 12 EG (0, F )=0. (34) If we replace coditio C4 with lim 12 h r+1 exists is greater tha 0, the % coverges to a ocetral ormal distributio. Proof. By Taylor's theorem, we have G (0, F )=G (0, F )&G (%, F ) =&D$G (0, F ) % +O(&% & 2 &D 3 2 (0, F )&). (35) By coditio C1 the law of iterated logarithms for U-statistics, we have D$G (0, F )=D$G (0, F)+O(h &:2 D 3 2 (0, F )=D 3 2 (0, F)+O(h &:2&1 a 12 ), w.p.1. (36) a 12 ), w.p.1. (37)

10 286 LIU AND RAO Further, by coditio C3, D$G (0, F)=ED 2 R q $(X 1,..., X k, %&h u) k(u) du %=0 =D 2 R q 2(%&h u, F) k(u) du %=0 Cosequetly = R q D 2 2(&h u, F) k(u) du =S(0, F)+o(h r ). (38) G (0, F )=&S(0, F) % +o(h r By the law of iterated logarithms, we have O the other h &% &)+O(h&:2 a 12 &% &) +O(&% & 2 )+O(h &:2&1 a 12 &% & 2 ), w.p.1 (39) S(0, F) % =&EG (0, F )+O(a 12 ), w.p.1. (40) EG (0, F )=D R q E$(X 1,..., X m, %&h u) k(u) du %=0 Therefore = D2(&h u, F) k(u) du=o(h r+1 ). (41) R q % =O(h r+1 )+O(a 12 )=O(a12 ), w.p.1, (42) (29) is proved. By the cetral limit theorem of U-statistics, we have 12 7 &12 (G (0, F )&EG (0, F )) N(0, m 2 I q_q ), i dist., (43) (30) is proved. 4. APPLICATIONS We apply the asymptotic theory of +-estimatio established i previous sectios to the media where p=1 m=1 to the media of Liu (1990) usig a smoothed DM.

11 +-ESTIMATION Smoothed Media Estimator It is well kow that the L 1 -orm estimator that miimizes : i=1 X i &% (44) is a sample media. Asymptotic properties of a sample media have bee well studied. We use this example to demostrate how well the kerel smoothig method performs. I this case, the observatios are i.i.d. uivariate rom variables m=1. Without loss of geerality, let us assume that the media is 0. Note that the smoothed discrepacy fuctio becomes $ (X, %)= R X&%+h u h(u) du. (45) Deote K(x)= x & k(x) dx. The we have D$ (X, %)=2K \%&X &1 (46) h + G (%, F )= : i=1\ 2K \ %&X i h + &1 +. (47) Theorem 4. Assume that D1. f(0)>0 f(x) is differetiable i a eighborhood of 0. D2. k(u) is twice differetiable has a compact support, D3. Choose the bwidth as The 1. % is strogly cosistet, i.e., R uk(u) du=0. (48) h =O( &13 (log log ) 13 ). (49) lim % =0, w.p.1. (50)

12 288 LIU AND RAO 2. % is asymptotically ormal, i.e. 2f(0) 12 % N(0, 1), i dist., as. (51) 3. % ca be represeted as % = 1 2f(0) : i=1\ 1&2K \ &X i h ++ If both k(u) f(x) are symmetric, the +O(&56 (log log ) 56 ), w.p.1. (52) % = 1 2f (0) i=1\ : 1&2K \ &X i h ++ +O(&1 log log ), w.p.1, (53) where f (0) is the kerel estimator of f(0). Proof. We eed oly to verify coditios of previous theorems. We first verify coditio A4 i Theorem 1. Other coditios are easy to verify. Note that sup G (%, F )&G (%, F) =2 sup } 1 : i=1 K \%&X i h + & K \%&x R h + f(x) dx } =2 sup } R (F (%&h x)&f(%&h x)) k(x) dx } C sup x F (x)&f(x) C &12 (log log ) 12, w.p.1. (54) The last iequality is from the DvoretzkyKieferWolfowitz theorem (1956) o KolmogorovSimirov distace (Serflig, 1980). By Theorem 1, (50) is true. Note that S(0, F)= d 2 d% E X 1&% 2 %=0 = d 2 d% 2\ x>% (x&%) f(x) dx& x<% (x&%) f(x) dx +}%=0 = d d% (2F(%)&1) %=0=2f(0), (55)

13 +-ESTIMATION =var(d$ (X 1, 0))=4 var \K \ &X 1 =4 R K 2\ &x h + h ++ f(x) dx&4 K \ \&x R h + 2 f(x) dx + =8 F(&h u) K(u) k(u) du&4 \ R F(&h u) k(u) du + 2 R =2+4f $(0) h 2\ R u 2 k(u) K(u) du& R u 2 k(u) du ++o(h2 ). (56) All other coditios i Theorem 3 are satisfied. Note that :=1 r=1. The (51) (52) follow from Theorem 3. Whe both k(u) f(x) are symmetric, we have E : i=1\ 1&2K \ &X i h ++ =0 (57) : i=1\ 1&2K \ &X i h ++ = : i=1\ 2K \ % &X i h =2 : i=1 k \ &X i h + % + &2K \ &X i h ++ + : h i=1 k$ \&X i h +\ =2f (0) % +f $ (0) % 2 +O \ \ % 3 h + % h + 2+O \ % \ h , (58) where f (0) f $ (0) are the kerel estimators of f(0) f $(0), respectively. It is see that 1 i=1\ 1&2K \ &X i h ++ : (53) is proved. =O(&12 (log log ) 12 ), w.p.1, Remark 6. The DM is a bouded fuctio so that coditios o f(x) k(u) eable us to exchage the order of itegrals itegral derivative ivolved i the proof of the theorem.

14 290 LIU AND RAO Remark 7. Bahadur (1966) Kiefer (1967) foud a asymptotic represetatio of a sample quatile. The BahadurKiefer represetatio of a sample media is give as! =!+ 1 2f(!) (1&2F (!))+O( &34 (log log ) 34 ), w.p.1. (59) The remaider rate caot be improved. However, uder the same coditios as those of a BahadurKiefer represetatio, a smoothed media estimator ca have a represetatio with a remaider rate &56 (log log ) 56. If f(x) is r times differetiable, the remaider rate ca be &(4r+1)(4r+2) (log log ) (4r+1)(4r+2). We would ot claim that the smoothed media estimator is better tha the sample media because the variace of the smoothed media is slightly larger tha the variace of the sample media. If the desity has high eough differetiability, we ca choose a high degree kerel fuctio (the first d momets are 0). The biases variaces are compared i the followig: Smoothed Sample Media Bias O( &(4r+1)(4r+2) (log log ) (4r+1)(4r+2) ) Variace (2f(!)) &2 &1 +O( &(3d+1)(2d+1) (log log ) (3d+1)(2d+1) ) Sample Media Bias O( &34 (log log ) 34 ) Variace (2f(!)) &2 &1 +O( &32 (log log ) 32 ) It is worth otig that the smoothed sample media has, i fact, less bias. Remark 8. The smoothed media estimate provides a alterative computig method for the L 1 -orm method i liear models. A compariso study of computatios based o liear programmig the smoothed estimatio method will be reported i a future paper. Remark 9. The kerel smoothed L 1 -orm i liear models was studied i a paper by Liu Jag (1998). Remark 10. A large class of robust M-estimates are i fact kerels smoothed L 1 -orm estimates (Liu, 1999) Simplicial Media Estimator By the otio of data depth (Liu, 1990), the deepest poit, which is called the simplicial media of a multivariate distributio, is a geeralizatio of the media to a multivariate distributio. Estimatio of a simplicial

15 +-ESTIMATION 291 media falls withi the +-estimatio framework. Let A(x)=A(x 1,..., x p+1 ) be a closed simplex i R p with vertices x 1,..., x p+1, defie I B ( y)= {1 if y # B, 0 if y B. (60) A simplicial media DM is $(x, %)=&I A(x) (%). (61) Applyig the kerel smoothig method, we have a smoothed DM. $ (x, %)= R p&i A(x) (%&h u) k(u) du. (62) Theorem 5. Assume: E1. F(x) has a uique simplicial media which is assumed to be 0 for coveiece, f(x) is differetiable, S(0, F) is positive defiite. E2. k(u) has a compact support, twice differetiable, E3. h =O( &14 (log log ) 14 ). The: 1. % is strogly cosistet, i.e., 2. % ca be represeted as uk(u) du=0. (63) R p lim % =0, w.p.1. (64) % =S &1 (0, F) G (0, F )+O( &34 (log log ) 34 ), w.p.1. (65) 3. % is asymptotically ormal, i.e., 12 S(0, F) % N(0, (p+1) 2 7), i dist. as, (66) where 7 is the variacecovariace matrix of D(E($(X, 0) X 1 )). Remark 11. The asymptotic distributio we obtaied is the same as that of Arcoes et al.'s (1994). Our coditios are much milder, simpler, easier to verify.

16 292 LIU AND RAO Remark 12. The smoothed method has a high potetial i the computatio of simplicial medias. Proof. We eed to verify coditio A1 i Theorem 1 fid : r for the smoothed simplicial media estimator. By the coditios i the theorem, we kow that r=1. Note that E(D 2 i 1, j 1 $ (X, 0)D 2 i 2, j 2 $ (X, 0)) =h &4 R p R p Pr(&h u,&h v # A(X)) D 2 i 1, j 1 k(u) D 2 i 2, j 2 k(v) du dv =O(h &2 ), (67) E(D 3 i 1, j 1, l 1 $ (X, 0)D 3 i 2, j 2, l 2 $ (X, 0)) =h &6 R p R p Pr(&h u&h v # A(X)) D 3 i 1, j 1, l 1 k(u) D 3 i 2, j 2, l 2 k(v) du dv =O(h &4 ). (68) Therefore :=2, it is easy to show that lim sup F (x)&f(x) =0, w.p.1 (69) x for the multivariate case this leads to lim sup L } 1 : I(X i # L)&P(X # L) w.p.1, (70) }=0, i=1 where L belogs to the Borel field. Now we verify coditio A4 i Theorem 1. It is see that sup &G (%, F )&G (%, F)& % =sup &D2 (%, F )&D2 (%, F)& % sup % " \ I(u # A(X)) d(f _}}}_F ) & p+1 I(u # A(X)) d(f_}}}_f) Dk \ %&u p+1 + h + du "

17 +-ESTIMATION 293 sup % " I(u # A(X)) d(f _}}}_F ) & p+1 I(u # A(X)) d(f_}}}_f) Dk \ %&u h +" du p+1 "" " Dk \ u du sup h+" u " I(u # A(X)) d(f _}}}_F ) & I(u # A(X)) d(f_}}}_f) p+1 " p+1 0, w.p.1,. (71) The last expressio is from (70). Other coditios i Theorem 3 are easy to verify. This theorem is proved. REFERENCES M. A. Arcoes, Z. Che, E. Gie, Estimatios related to U-processes with applicatio to multivariate media: Asymptotic ormality, A. Statist. 22 (1994), R. R. Bahadur, A ote o quatiles i large samples, A. Math. Statist. 37 (1966), A. Dvoretzky, J. Kiefer, J. Wolfowitz, Asymptotic miimax character of the sample distributio fuctio of the multiomial estimator, A. Math. Statist. 27 (1956), W. Hoeffdig, A class of statistics with asymptotically ormal distributio, A. Math. Statist. 19 (1948), J. L. Horowitz, A smoothed maximum score estimator for the biary respose model, Ecoometrica 60 (1992), J. L. Horowitz, Bootstrap methods for media regressio models, Ecoometrica 66 (1998), P. J. Huber, Robust estimatio of a locatio parameter, A. Math. Statist. 43 (1964), J. Kiefer, O Bahadur's represetatio of sample quatiles, A. Math. Statist. 38 (1967), R. Y. Liu, O a otio of data depth based o rom simplicities, A. Statist. 18 (1990), Z. J. Liu M.-Y. Jag, Asymptotic theory of smoothed LAD estimates i liear regressio models, Iteratioal J. Math. Statist. Sci. 7 (1998), Z. J. Liu, A ote o M-estimatio smoothed L 1 -orm, mauscript, H. Oja, Descriptive statistics for multivariate distributio, Statist. Prob. Lett. 1 (1983), D. Pollard, ``Covergece of Stochastic Process,'' Spriger-Verlag, BerliNew York, R. J. Serflig, ``Approximatio Theorems for Mathematical Statistics,'' Wiley, New York, 1980.