Second Order Hadamard Differentiability in Statistical Applications

Transkript

1 Joural of Multivariate Aalysis 77, (21) doi:1.16jmva , available olie at o Secod Order Hadamard Differetiability i Statistical Applicatios Jia-Jia Re 1 Tulae Uiversity Praab Kumar Se Uiversity of North Carolia at Chapel Hill Received November 17, 1998; published olie February 5, 21 A formulatio of the secod-order Hadamard differetiability of (exteded) statistical fuctioals some related theoretical results are established. These results are applied to derive the limitig distributios of a class of geeralized Crame rvo Mises type test statistics, which iclude some proposed ew oes for the tests of goodess of fit i the 3-sample problems, the tests i liear regressio models, the tests of bivariate idepedece, as special cases. 21 Academic Press AMS 1991 subject classificatios: 62E2; 62G1; 62G99; 62J5. Key words phrases: Crame rvo Mises statistic; goodess of fit test; Hadamard differetiability; limitig distributio; liear models; M-estimator; statistical fuctioals; uiform asymptotic liearity; weighted empirical processes. 1. INTRODUCTION Let X 1,..., X be idepedet rom variables each havig distributio fuctio (d.f.) F 1,..., F, respectively, let c 1,..., c be a sequece of real umbers. Cosider a weighted empirical process S*(x)=C &1 : c i I[X i x], where c i =(c i &c )C, c = &1 c i, C 2 = (c i&c ) 2, cosider S (x)=(+1) &1 : 187 I[X i x]. 1 The author's research was partially supported by NSF grats DMS DMS X1 35. Copyright 21 by Academic Press All rights of reproductio i ay form reserved.

2 188 REN AND SEN Motivated by the pioeerig work of Ha jek (1963), we are iterested i the followig fuctioal of (S *, S ), 9(S*, S )= [S *(x)] 2 (S (x)) ds (x), (1.1) & where is a o-egative weight fuctio. The followig example shows oe applicatio of this fuctioal i statistics problems. Example 1 (Tests i Simple Liear Regressio Model). Let X i 's be the observatios of the followig simple liear regressio model, X i =:+c i ;+= i,,...,, (1.2) where c i are kow regressio costats, (:, ;) is the vector of ukow (regressio) parameters to be estimated, = i are idepedet idetically distributed rom variables (i.i.d.r.v.). For testig the ull hypothesis H ; ;= vs H 1 : ;{, (1.3) we have that with #1, C 2 9(S *, S ) is equivalet to the Crame rvo Mises type test statistic (Ha jek S8 ida k, 1967, p. 13). We refer to Ha jek S8 ida k (1967) for some geeral motivatio a useful survey of the related asymptotic theory. For a geeral weight fuctio, 9(S *, S )is termed as geeralized Crame rvo Mises (GCvM) type statistic its asymptotic properties have ot bee studied i the literature. The motivatio of such a use of the weight fuctio i certai situatios is give i Example 3 of Sectio 2, which shows that the use of (t)=[t(1&t)] &1 is of special importace. Several additioal examples o the applicatios of fuctioal 9(S *, S ) i statistical problems are preseted i Sectio 2, where some ew test statistics are proposed for the tests of goodess of fit i the 3-sample problems, the tests i liear regressio models the tests of bivariate idepedece. For all these examples, asymptotic distributio theory of 9(S *, S )isof focal importace. Thus we ited to develop a geeral approach for such studies i this paper. Let F be a cotiuous d.f., observe that 9 iduces a fuctio defied o the space D[, 1]_D[, 1], {(U*, U )= 1 [U *(t)] 2 (U (t)) du (t), (1.4) where U *=S * b F &1, U =S b F &1 D[, 1] is the space of right cotiuous real valued fuctios with left h limits edowed with the

3 SECOND ORDER HADAMARD DIFFERENTIABILITY 189 supremum orm &}&. I Example 1, F is the commo d.f. of X i 's uder H. We will see later o that this fuctioal { is Hadamard differetiable at (, U), where U is the uiform d.f. o [, 1]. Hece, we have a form of the Taylor expasio, {(U *, U )={(, U)+{$ (, U) (U *, U &U)+Rem(U *, U &U; {), (1.5) where {$ (, U) is a liear fuctioal is the Hadamard derivative at (, U), Rem(U *, U &U; {) is the remaider term of this first-order expasio. Usually, we would expect to derive the asymptotic ormality of {(U *, U ) through verifyig C Rem(U *, U &U; {) w P, as. (1.6) Refereces o this approach ca be foud i Re (1994) Re Se (1995) or a simpler case i Ferholz (1983). However, we have {$ (, U) # i (1.5) for our examples. Hece, the first-order expasio (1.5) caot help us obtai the limitig distributio of {(U *, U )=9(S *, S ). As oted i vo Mises (1947), this leads us to cosider a higher order expasio of (1.5), i.e., {(U*, U )={(, U)+{$ (, U) (U *, U &U)+ 1 {" 2 (, U)(U *, U &U) +Rem 2 (U *, U &U; {), (1.7) where {" (, U) is the secod-order Hadamard derivative at (, U) Rem 2 (U *, U &U; {) is the remaider term of this secod-order expasio. It appears that this secod-order expasio (1.7) also provides some deeper asymptotic results o regressio M-estimators i liear models. Such a applicatio is discussed i Example 5 of Sectio 2. Moreover, the bivariate versio of (1.7), i.e., for bivariate rom vectors X i =(V i, W i ) (see Re Se, 1995, for the first-order expasio with multivariate rom vectors), gives a coveiet tool to study the limitig distributios of the test statistics for the tests of idepedece of V i W i, which is briefly discussed i Example 6 of Sectios 2 5. Motivated by the studies described above by broader potetial applicatios i other studies of the asymptotic properties of the statistics i differet model settigs (for istace, the L- R-estimators i liear model, etc.), the cocept of secod-order Hadamard differetiability (SOHD) some related theoretical results are established i Sectio 3 with proofs deferred to Sectio 6. While there might be methods other tha the SOHD method to study the asymptotic distributios of the statistics cosidered i this paper, the geeral motivatio of our work here is that the SOHD property possessed by a statistic provides more iformatio tha the first-order Hadamard differetiability property about the statistic's

4 19 REN AND SEN asymptotic behavior. Thereby, deeper asymptotic properties of the statistics ca be obtaied more easily. I some situatios (e.g., Examples 24 6 of Sectio 2), we may hope that the testig or estimatig procedure ca be costructed studied coveietly by the SOHD method. The applicatios of the cocept of SOHD are cosidered i Sectio 4 i derivig the limitig distributios of {(U *, U ) give by (1.4) with proofs deferred to Sectio 6. These results are used i Sectio 5 to study the specific limitig distributios of the test statistics give i Examples 14 6 of Sectios 1 2. The use of the special weight fuctio (S )= [S (1&S )] &1 is icluded as a special case i our studies. 2. EXAMPLES I the followig examples, the GCvM type statistics 9(S *, S ) are used as test statistics, some of which are ew or have ot bee studied i the literature. Example 2 (Tests i Geeral Liear Regressio Model). We geeralize the Crame rvo Mises type test statistic by Ha jek S8 ida k (1967) to a geeral liear model. Let X i 's be the observatios of the followig liear regressio model, X i =:+c i ;+d i #+= i,,...,, (2.1) where c i, d i are kow regressio costats, (:, ;, #) is the vector of ukow (regressio) parameters to be estimated, = i are i.i.d.r.v.'s. Let d i =(d i &d )D, d = &1 d i,d 2 = (d i&d ) 2, let T *(x)=d &1 For testig the ull hypothesis : d i I[X i x]. (2.2) H :(;, #)= vs H i :(;, #){, (2.3) proceedig as i Ha jek S8 ida k (1967), we propose the followig fuctioals of (S *, T *, S ): & (C S *(x), D T *(x))(c S *(x), D T *(x)) T (S (x)) ds (x) =C 2 9(S *, S )+D 2 9(T *, S ) (2.4)

5 SECOND ORDER HADAMARD DIFFERENTIABILITY 191 or max 2 {C [S*(x)] 2 (S (x)) ds (x), D 2 [T*(x)] 2 (S (x)) ds (x) = & & =max[c 2 9(S *, S ), D 2 9(T *, S )] (2.5) to be used as GCvM type test statistics. Oe may ote that through rak statistics C 2 9(S *, S )D 2 9(T *, S ) measure the effects that c i 's d i 's have o X i 's, respectively. Clearly, these statistics ca be easily further exteded to the liear regressio model with p ukow regressio parameters, which will be briefly discussed i Sectio 5. Example 3 (Goodess of Fit Tests i 3-Sample Problem). statistic give i (2.4) may be used for the 3-sample problem. The test (a) Case with equal sample size. For =3m, let X i,,..., m, have d.f. F, X i, i=m+1,..., 2m, have d.f. G, X i, i=2m+1,...,, have d.f. H. We are iterested i the followig 3-sample problem Suppose that H : F=G=H vs H 1 : H ot true. (2.6) & 1 3 Q=_a 1 a 2 a b 1 b 2 b 3 is a orthogoal matrix. I model (2.1), if we let (a 1, b 1 ),,..., m (c i, b i )={(a 2, b 2 ), i=m+1,..., 2m (2.7) (a 3, b 3 ), i=2m+1,...,, the model (2.1) becomes :+a 1 ;+b 1 #+= i, X i ={:+a 2 ;+b 2 #+= i, :+a 3 ;+b 3 #+= i,,..., m i=m+1,..., 2m i=2m+1,...,

6 192 REN AND SEN H of the 3-sample problem (2.6) implies H : a 1 ;+b 1 #=a 2 ;+b 2 #= a 3 ;+b 3 #, which is equivalet to test (2.3) because of a 2 1 +a2 2 +a2 3 =b2 1 + b 2 2 +b2 3 =1 a 1+a 2 +a 3 =b 1 +b 2 +b 3 =a 1 b 1 +a 2 b 2 +a 3 b 3 =. From (2.7), we have C 2 =D2 =m, (a 1 - m, b 1 - m),,..., m (c i, d i )={(a 2 - m, b 2 - m), i=m+1,..., 2m (a 3 - m, b 3 - m), i=2m+1,..., C S *(x)=a 1 - mf m (x)+a 2 - mg m (x)+a 3 - mh m (x) D T *(x)=b 1 - mf m (x)+b 2 - mg m (x)+b 3 - mh m (x), where F m, G m H m are the empirical d.f.'s for (X 1,..., X m ), (X m+1,..., X 2m )(X 2m+1,..., X 3m ), respectively. Thus, we have \ +1 - S, C S *, D T * + T =Q(- mf m, - mg m, - mh m ) T C 2 S*2 +D2 T*2 =m(f 2 m +G2 +H 2 )& m m \ +1 2 S + =m[(f 2 m +G2 m +H 2 m )&3[(F m+g m +H m )3] 2 ] =m {\F m& +1 + \H m& +1 2 S + 2 S + =. + \G m& +1 2 S + Hece, from (2.4), the test statistic for the 3-sample problem (2.6) is give by C 2 9(S *, S )+D 2 9(T *, S ) =m &{\ F m& +1 2 S + + \G m& +1 2 S + + \H m& +1 2 S + = (S ) ds. (2.8) With #1 (t)=[t(1&t)] &1, the test statistics (2.8) are equivalet to those by Kiefer (1959) Scholz Stephes (1987), respectively. We

7 SECOND ORDER HADAMARD DIFFERENTIABILITY 193 ote that uder H, the asymptotic variace of - m[f m (x)& +1 S (x)] is give by 2 3F(x)(1&F(x)). I the oe-sample goodess of fit tests, Aderso Darlig (1952) used the reciprocal of the asymptotic variace of - m[f m (x)&f(x)] uder the ull hypothesis as the weight fuctio to put equal weight (i a certai sese) to each poit of the distributio. However, i the 3-sample goodess of fit test (2.6), F is ot kow eve uder H. To geeralize this idea of usig the weight fuctio from oe-sample tests to 3-sample tests, we may use the reciprocal of the estimator for the asymptotic variace of - m[f m (x)& +1 S (x)] as the weight fuctio, i.e., we may use (S (x))=[s (x)(1&s (x))] &1 as a weight fuctio. Note that for this special weight fuctio, 9(S*, S ) i (1.1) is well defied because of our choice of S. For a geeral weight fuctio, (2.8) gives GCvM type test statistic for the 3-sample goodess of fit tests. (b) Case with o-equal sample size. Suppose that for = , the rom samples (X 1,..., X 1 ), (X 1 +1,..., X ) (X ,..., X ), are draw from F, G H with the empirical d.f.'s F 1, G 2 H 3, respectively. Let Q =_a 1 a 2 a 3& b 1 b 2 b 3 be a orthogoal matrix, let C 2 =D2 = with The, we have (a 1-1, b 1-1 ),,..., 1 (c i, d i )={(a 2-2, b 2-2 ), i= 1 +1,..., (a 3-3, b 3-3 ), i= ,...,. \ +1 - S, C S *, D T * + T =Q (- 1 F 1, - 2 G 2, - 3 H 3 ) T, C 2 S*2 +D2 T*2 = 1\ F 1 & +1 2 S + + 2\ G 2 & +1 2 S + + 3\ H 3 & +1 2 S +,

8 194 REN AND SEN from (2.4), we have the followig test statistic for (2.6) with o-equal sample size C 2 9(S *, S )+D 2 9(T *, S ) = &{ 1\ F 1 & +1 2 S + + 2\ G 2 & +1 2 S + + 3\ H 3 & +1 2 S + = (S ) ds. (2.9) Example 4 (Alterative Tests i 3-Sample Problem). Fuctioals (2.4) (2.5) ca be used to costruct alterative tests for the 3-sample problem (2.6). Cosider a more geeral case of model (2.1), deote X i =:+c i ;+d i #+e i '+= i,,...,, (2.1) R *(x)=e &1 : e i I[X i x], where = i =(e i &e )E, e = &1 e i, E 2 = (e i&e ) 2. Several alterative test statistics for (2.6) are give as follows. (a) I (2.1), let (1,, ),,..., m (c i, d i, e i )=_(, 1, ), i=m+1,..., 2m (2.11) (,, 1), i=2m+1,..., ; the model (2.1) becomes a special case of the 3-sample problem with equal sample size, H of the goodess of fit test (2.6) implies H : ;= #='. Sice uder this H, X i 's are i.i.d., from Ha jek S8 ida k (1967, see discussio o p. 9), we kow that the test statistics (2.4) or (2.5) may be used here. For (2.11), we have C 2 =D 2 =E 2 = 2 3 m (C S *, D T *, E R *) =- 3m2 \\F m& +1 S +, \ G m& +1 S +, \ H m& +1 S ++,

9 SECOND ORDER HADAMARD DIFFERENTIABILITY 195 from (2.5), we have the followig test statistic for (2.6): max[c 2 9(S *, S ), D 2 9(T *, S ), E 2 9(R *, S )] = 3 2 max { m &\ F m& +1 2 S + (S ) ds, m &\ G m& +1 2 S + (S ) ds, m &\ H m& +1 2 S + (S ) ds =. (2.12) For a geeral weight fuctio, this statistic has ot bee studied; for the case of #1, it is equivalet to that give by Kiefer (1959), where the limitig distributio of this test statistic was ot derived. (b) For the case of the 3-sample problem with o-equal sample size, let i (2.1) C &1 \ (c i, d i, e i )={D &1 \ E &1 \ 1& 1,& 2,& 3 +,,..., 1 & 1,1& 2,& 3 +, i= 1+1,..., & 1,& 2,1& 3 + i= ,..., (2.13) C 2 = 1\ 1& 1 +, D2 = 2\ 1& 2 +, E 2 = 3\ 1& 3 +. The, we have C S *= 1 & 1 \ D T *= 2 & 2 \ E R *= 3 & 3 \ F 1 & +1 S + G 2 & +1 S +, H 3 & +1 S +,

10 196 REN AND SEN from (2.5), we have the followig test statistic for the 3-sample problem (2.6) with o-equal sample size max[c 2 9(S *, S ), D 2 9(T *, S ), E 2 9(R *, S )] =max { 1 & 1 &\ F 1 & +1 2 & 2 &\ G 2 & +1 2 S + (S ) ds, 3 & 3 &\ H 3 & +1 2 S + (S ) ds, 2 S + (S ) ds =. (2.14) (c) I (2.1), let (1,,&1),,..., m (c i, d i, e i )={(&1, 1, ), i=m+1,..., 2m (2.15) (, &1, 1) i=2m+1,..., ; the C 2 =D2 =E 2 =2m (C S *, D T *, E R *)=- m2 ((F m &G m ), (G m &H m ), (H m &F m )), model (2.1) becomes a special case of the 3-sample problem with equal sample size, H of the goodess of fit test (2.6) implies H : ;=#='. From (2.5) (2.4), we have the followig test statistics for (2.6) max[c 2 9(S *, S ), D 2 9(T *, S ), E 2 9(R *, S )] = 1 2 max {m (F m &G m ) 2 (S ) ds, & m (G m &H m ) 2 (S ) ds, & m (H m &F m ) 2 (S ) ds & =, (2.16) C 29(S *, S )+D 29(T *, S )+E 29(R *, S ) = m 2 [(F m &G m ) 2 +(G m &H m ) 2 +(H m &F m ) 2 ](S ) ds, & (2.16a)

11 SECOND ORDER HADAMARD DIFFERENTIABILITY 197 respectively. For a geeral weight fuctio, these statistics have ot bee studied; for the case of #1, David (1958) costructed studied a test statistic which is the KolmogorovSmirov versio of (2.16). (d) For the case of the 3-sample problem with o-equal sample size, let i (2.1) C &1(- 2 1,,&- 3 1 ),,..., 1 (c i, d i, e i )={D &1 (&- 1 2, - 3 2,), i= 1 +1,..., E &1 (, &- 2 3, ), i= ,...,, (2.17) the we have C 2 = 1 + 2, D 2 = 2 + 3, E 2 = ; C S *= (F 1 &G 2 ), D T *= (G 2 &H 3 ), E R *= (H 3 &F 1 ), from (2.5) (2.4), we have the followig test statistics for (2.6) with o-equal sample size respectively. max[c 2 9(S *, S ), D 2 9(T *, S ), E 2 9(R *, S )] =max { (F 1 &G 2 ) 2 (S ) ds, & (G 2 &H 3 ) 2 (S ) ds, & (H 3 &F 1 ) 2 (S ) ds & =, (2.18) C 2 9(S *, S )+D 2 9(T *, S )+E 2 9(R *, S ) = &{ (F 1 &G 2 ) (G 2 &H 3 ) (H 3 &F 1 ) 2= (S ) ds, (2.18a)

12 198 REN AND SEN Clearly, our test statistics i Examples 34 here ca be easily exteded to treat the k-sample problem (k2). The ext example describes the applicatio of the secod-order expasio (1.7) for statistical fuctioals to the studies of deeper asymptotic results o regressio M-estimators i liear models. Example 5 (Regressio M-estimators). For simplicity of presetatio, we cosider the simple liear regressio model give by (1.2) with error distributio F. The robust M-estimator of (:, ;), deoted by (:^, ; ), is give by a solutio (with respect to (% 1, % 2 )) of the estimatig equatios : : (X i &% 1 &c i % 2 )= c i (X i &% 1 &c i % 2 )=, (2.19) where : R R is a suitable score fuctio. Settig Y i =X i &:&c i ; (i.i.d.r.v.'s with d.f. F), 1i, with c i=, u=(u 1, u 2 ) T # R 2 for u 1 =- (% 1 &:), u 2 =C (% 2 &;), we have that (2.19) is equivalet to M 1 (u)= : M 2 (u)= : &12 (Y i &c T i u)= (2.2) c i (Y i &c T i u)=, where c i =( &12, c i ) T. Lettig M (u)=(m 1, M 2 ) T, the asymptotic ormality related properties of (:^, ; ) have bee studied most coveietly by icorporatig the followig uiform asymptotic liearity for the M-estimators: sup M (u)&m ()+#Q u w P, as, (2.21) u K where K is ay fiite positive real umber, } sts for supremum orm o R 2, #=$ df>, Q = c ic T i. Note that M (u) is a liear fuctioal of the empirical fuctios V (t, u)= : V*(t, u)= : &12 I[Y i F &1 (t)+c T i u] c i I[Y i F &1 (t)+c T i u],

13 SECOND ORDER HADAMARD DIFFERENTIABILITY 199 where t # [, 1] u # R 2, viz., M (u)={ L (V (},u), V *( }, u)) = \ b F &1 dv (}, u), b F &1 dv *( b, u) + T, that M (u) is a fuctioal defied o D[, 1]_D[, 1], because V (},u) V *( }, u) are elemets o D[, 1] for ay fixed u # R 2. Sice the Hadamard derivative is a liear fuctioal, M (u) could be the Hadamard derivative of some appropriate fuctioal {. The choice of { may ot be uique, geeral motivatios for this are give i Re Se (1991). Re Se (1991) showed that if a fuctioal { is Hadamard differetiable with first-order derivative M (u), the as sup u K }_ - a & [{(V (}u), V *( }, u))]&[m (u)&m ()] } wp, (2.22) where a = c+ i = c& i with c+ i =max[, c i] c & i =&mi[, c i], for some fuctioal { 1, { 2 : D[, 1] R, {(V (},u), V *( }, u))=({ 1 (V (},u)- ), { 2 (V *( }, u)a )) T. (2.23) Thereby, usig a coveiet fuctioal {, Re Se (1994) established (2.21) from showig sup u K }_ - a & [{(V (},u), V *( }, u))]+#q u } wp, as. This (first-order) Hadamard differetiability approach for establishig (2.21) compares quite favorably with the alterative oes i the literature (Re Se, 1994). Note that (2.22) is give by the asymptotic behavior of Rem(V (}, u)&u, V *( }, u)&u; {). Hece, we aturally expect to obtai more detailed asymptotic properties of (2.21): the covergece rate i probability, usig the secod-order expasio (1.7) for the fuctioal { i (2.22) (V (},u), V *( }, u)). I this cotext, some asymptotic results o Rem 2 (V (},u)&u, V *( }, u)&u; {) will be give i Sectio 3. These results have bee used to establish a asymptotic represetatio of (:^, ; ) uder weaker coditios tha those available i the literature (Re Se, 1993).

14 2 REN AND SEN Example 6 (Test of Idepedece). Let (V 1, W 1 ),..., (V m, W m ) be a rom sample from a bivariate d.f. F(v, w). If oe wishes to test if V i W i are idepedet, the test statistic for the followig hypothesis, H : F(v, w)=f(v, ) F(, w) H 1 : F(v, w){f(v, ) F(, w), (2.24) may be give by the bivariate versio of (1.1). To see this, let =m+m 2 for 1i, deote X i = {(V i, W i ) (V i& jm, W j ) x=(v, w), if 1im if jm+1i(j+1) m, j=1,..., m (2.25) I[X i x]=i[v i v, W i w]. Thus, i (1.1) for c i =2, 1im; c i =1, m+1i, we have C 2 = m 2 (m+1), S*(x)=C &1 : c i I[X i x]=[f m (v, w)&f m (v, ) F m (, w)] (2.26) S (x)= +1_ 1 m+1 F m(v, w)+ m m+1 F m(v, ) F m (, w) &, where F m is the bivariate empirical d.f. of (V 1, W 1 ),..., (V m, W m ). Hece, the bivariate versio of (1.1) gives a test statistic for (2.24): vs C 2 9(S *, S )=C 2 [S *(x)] 2 (S (x)) ds (x) = m+1 m2 [F m(v, w)&f m (v, ) F m (, w)] 2 _(S (x)) ds (x). (2.27) We ote that although the rom vectors X i i (2.25), 1i, are ot all idepedet from oe other, S *(x), S (x) are bivariate processes, a more geeral multivariate form of (1.5) for the first-order Hadamard derivative is studied by Re Se (1995), the multivariate aalogue of (1.7) for the secod order Hadamard derivative applies to the statistic give i (2.27), because uder H, - ms *(x) weakly coverges to a cetered Gaussia process S (x) uiformly coverges to F(v, ) F(, w) with probability 1. This will be briefly discussed i Sectio 5. Hoeffdig (1948) studied the idepedece test (2.24) usig

15 SECOND ORDER HADAMARD DIFFERENTIABILITY 21 U-statistics (its limitig distributio uder H was ot specifically give), while our formulatio i this paper does ot require U-statistics represetatio directly coects the degeerated limitig distributio of the test statistic uder H with the secod-order Hadamard derivative. 3. SECOND-ORDER HADAMARD DIFFERENTIABILITY First-order Hadamard differetiability is usually defied as follows. Let V W be the topological vector spaces, L 1 (V, W) be the set of cotiuous liear trasformatios from V to W, A be a ope set of V. Defiitio I. A fuctioal T: A W is Hadamard differetiable (or compact differetiable) atf # A if there exists T$ F # L 1 (V, W) such that for ay compact set 1 of V, lim t T(F+tH)&T(F)&T$ F (th) = (3.1) t uiformly for ay H # 1. The liear fuctio T$ F is called the Hadamard derivative of T at F. For the sake of coveiece, i (3.1) we usually deote Rem 1 (th)=t(f+th)&t(f)&t$ F (th) (3.2) as the remaider term of the first-order expasio. This defiitio is related to the origial oe give i Reeds (1976) (see Ferholz, 1983). We should ote that i ormed vector spaces, (3.1) is equivalet to the followig form (viz., Gill, 1989) lim t Rem 1 (F+tH ) =, (3.1a) t for ay sequeces H with H H # V. Let C(V, W) be the set of cotiuous trasformatios from V to W, let L 2 (V, W )=[f; f # C(V, W ), f(th)=t 2 f(h) for ay H # V, t # R]. (3.3) We defie secod-order Hadamard differetiability as follows.

16 22 REN AND SEN Defiitio II. A fuctioal T: A W is secod-order Hadamard differetiable at F # A if there exists T$ F # L 1 (V, W ) T" F # L 2 (V, W ) such that for ay compact set 1 of V T(F+tH)&T(F)&T$ F (th)& 1 lim T" 2 F(tH) = (3.4) t t 2 uiformly for ay H # 1. T$ F T" F are called the first- secod-order Hadamard derivatives of T at F, respectively. We deote the remaider term of the secod-order expasio as below: Rem 2 (th)=t(f+th)&t(f)&t$ F (th)& 1 2 T" F(tH). (3.5) I ormed vector spaces, (3.4) may be preseted i a equivalet form, Rem 2 (F+tH ) lim =, (3.4a) t t 2 for ay sequeces H with H H # V. Remark 1. I the literature, various types of higher order derivatives have bee cosidered by some authors, such as vo Mises (1947), Averbukh Smolyaov (1967), Keller (1974), Reeds (1976), Se (1988), amog others. Averbukh Smolyaov (1967) defied the higher order derivative iductively; that is the secod-order derivative is defied if the map F # A T$ F # L 1 (V, W ) is differetiable. I Keller (1974) Reeds (1976), the secod-order derivative is required to be a ``2-liear map''. Oe may ote that all these defiitios require the fuctioal T to be at least cotiuously differetiable at F, while our defiitio of the secod-order Hadamard derivative i Defiitio II does ot require this, thus is a weaker differetiability coditio. Examples show that i some situatios, cotiuous differetiability coditio fails to hold (see Gill, 1989). Oe may also ote that based o our Defiitio II, the computatio of the secod-order Hadamard derivative is more straightforward. I (2.2) of Se (1988), if we let T$ F (H)= T 1(F; x) dh(x) T" F (H)= T 2(F; x, y) dh(x) dh( y), the (2.2) of Se (1988) coicides with our (3.5) for cotiuous bouded T 1 T 2. Later o oe will see that our cocept of secod-order Hadamard differetiability i Defiitio II suffices i our studies here.

17 SECOND ORDER HADAMARD DIFFERENTIABILITY 23 Remark 2. From our defiitio of secod-order Hadamard differetiability, it is obvious that the existece of the secod-order Hadamard derivative implies the existece of the first-order Hadamard derivative, we have T$ F ($ x &F)=IC(x; F, T )= d dt T(F+t($ x&f)) t= (3.6) T" F ($ x &F)= d 2 where $ x is the d.f. of the poit mass oe at x. dt 2 T(F+t($ x&f)) t=, (3.7) It is kow that the chai rule holds for first-order Hadamard differetiability (Ferholz, 1983), which makes it useful. We will show that the chai rule also holds for secod-order Hadamard differetiability. The proof is give i Sectio 6. Propositio 3.1. Let V, W Z be the topological vector spaces with T: V W Q: W Z. If T is secod-order Hadamard differetiable at F # V if Q is secod-order Hadamard differetiable at T(F)#W, the {=Q b T is secod-order Hadamard differetiable at F with derivatives {$ F =Q$ T(F) b T$ F (3.8) {" F =(Q b T )" F =Q" T(F) b T$ F +Q$ T(F) b T" F. (3.9) I our curret study, we will be primarily iterested i the fuctioals defied o Baach space (D[, 1]_D[, 1], &}&), where &}& sts for the supremum orm the _-field geerated by all ope balls is equipped (see Shorack Weller, 1986, for refereces). I the ext few propositios, we give some sufficiet coditios for a secod-order Hadamard differetiable fuctioal defied o the space D[, 1] or D[, 1]_D[, 1] with the proofs deferred to Sectio 6. The ext propositio is a geeralizatio of Propositio of Ferholz (1983) for secod-order Hadamard differetiability. Propositio 3.2. Let L: R R be differetiable L$ be cotiuous, bouded piecewise differetiable with a bouded derivative. Let #: D[, 1] L p [, 1], p1, be defied by #(H)=L b H, H # D[, 1]

18 24 REN AND SEN let A be the set of poits i R where L$ is ot differetiable. If # is defied i a eighborhood of Q # D[, 1] if +[x; Q(x)# A]=, where + is Lebesgue measure, the # is secod-order Hadamard differetiable at Q with derivatives #$ Q (H)=(L$ b Q) H #" Q (H)=(L" b Q) H 2, H # D[, 1]. Propositio 3.3. Let #: D[, 1]_D[, 1] D[, 1] be defied by #(G, H)=G,(H), G, H # D[, 1] where, is a real valued fuctio with cotiuous secod derivative,". The for G, H # D[, 1], # is secod-order Hadamard differetiable at (G, H ) with derivatives #$ (G, H )(G, H)=G,$(H ) H+,(H ) G, G, H # D[, 1] #" (G, H )(G, H)=2,$(H ) GH+G,"(H ) H 2, G, H # D[, 1]. We should otice that a special case of the above propositio with G # requires weaker coditios o,. We state this case as a corollary without proof. Corollary 3.4. Let #: D[, 1]_D[, 1] D[, 1] be defied by #(G, H)=G,(H), G, H # D[, 1], where, is a real valued fuctio with cotiuous derivative,$. The for H # D[, 1], # is secod-order Hadamard differetiable at (, H ) with derivatives #$ (, H )(G, H)=,(H ) G #" (, H )(G, H)=2,$(H ) GH, G, H # D[, 1]. The proof of the followig propositio, give i Sectio 6, is similar to that of Lemma 3 by Gill (1989), where a class of elemets i D[, 1]_ D[, 1] is cosidered, E= {(G, H); G, H # D[, 1] with 1 dh C =, (3.1) for a positive costat C. For detailed discussios o E, see Gill (1989).

19 SECOND ORDER HADAMARD DIFFERENTIABILITY 25 Propositio 3.5. Let #: D[, 1]_D[, 1] R be defied by #(G, H)= 1 G 2 (t) dh(t), G, H # D[, 1], let (G, H )#E with 1 dg <. The # is secod-order Hadamard differetiable at (G, H ) with derivatives #$ (G, H )(G, H)=2 1 G(t) G (t) dh (t)+ 1 G, H # D[, 1], G 2 (t) dh(t), #" (G, H )(G, H)=2 1 G 2 (t) dh (t)+4 1 G(t) G (t) dh(t), G, H # D[, 1]. For U* U give i Sectio 1, the secod-order remaider term Rem 2 i (1.7) has the followig results which are similar to those for the first-order remaider term. The proofs are give i Sectio 6. Let C[, 1] be the space of real valued cotiuous fuctios edowed with the supremum orm &}&. We impose the followig assumptios o U *U throughout this paper. Assumptio A. (A1) For some U* U # C[, 1], we have that as, C [E[U *]&U*] ww &}& - [E[U ]&U ] ww &}& ; (A2) &1 C 2 M, for all 1 some <M<; (A3) lim max 1i [c 2 ]=. i Theorem 3.6. Suppose {: D[, 1]_D[, 1] R is a fuctioal. The, uder Assumptio A, (i) if { is Hadamard differetiable at (U*, U ), we have C Rem 1 (U *&U*, U &U ; {) w P, as, (3.11) C [{(U *, U )&{(U*, U )]=C {$ (U*, U ) (U *&U*, U &U )+o p (1); (3.12) (ii) if { is secod-order Hadamard differetiable at (U*, U ), we have C 2 Rem 2(U *&U*, U &U ; {) w P, as, (3.13)

20 26 REN AND SEN C 2 [{(U *, U )&{(U*, U )]=C 2 {$ (U*, U )(U *&U*, U &U ) + 1 C 2 {" 2 (U*, U )(U *&U*, U &U )+o p (1). (3.14) Referrig to Example 5 i Sectio 2, we have the followig theorem o the secod-order remaider term Rem 2 (V (},u)&u, V *( }, u)&u; {) for the fuctioal { i (2.22). Let V* + (t, u)= : c + I[Y i if &1 (t)+c T iu] (3.15) (t, u)= : V* & the for this example, we usually have c & I[Y i if &1 (t)+c T iu]; (3.16) { 2 (V *( }, u)a )={ 1 (V* + (},u)a )&{ 1 (V* & (},u)a ) (3.17) i (2.23) (see Re Se, 1993). Theorem 3.7. Suppose { 1 : D[, 1] R is a fuctioal is secodorder Hadamard differetiable at U. Assume that F is absolutely cotiuous with a positive uiformly cotiuous derivative. The, for ay K>, (i) we have sup Rem 2 ( &12 V (}, u)&u; { 1 ) w P, as ; (3.18) u K (ii) uder (A3), we have sup a 2 Rem 2(a &1 V* + (},u)&u; { 1 ) w P, as u K sup u K a 2 Rem 2(a &1 V* & (},u)&u; { 1 ) w P, as. The proof of Theorem 3.7 is similar to that of Theorem 3.1 i Re Se (1991), where we oly eed to replace t &1 Rem(tH; { 1 ) by t &2 Rem 2 (th; { 1 ). Oe may ote that the Skorohod topology was used i

21 SECOND ORDER HADAMARD DIFFERENTIABILITY 27 Re Se (1991), but their results apply here ( i Sectio 6) because the limitig Gaussia process for the weak covergece here i Re Se (1991) has cotiuous sample path (see discussios i Gill, 1989) the use of the _-field geerated by all ope balls i this paper esures the measurability of the weighted empirical processes. Remark 3. Re Se (1993) have used Theorem 3.7 alog with Propositio 3.2 to study the covergece rate i probability of (2.21). Thereby, a asymptotic represetatio of M-estimators i liear models is give uder weaker coditios o the score fuctio, the error distributio F the desig matrix tha those i Jure ckova Se (1984). 4. LIMITING DISTRIBUTIONS OF {(U*, U ) I this sectio, we will cosider the fuctioal { give by (1.4). Let {: D[, 1]_D[, 1] R be a fuctioal, give by {(G, H)= 1 [G(x)] 2 (H(x)) dh(x), G, H # D[, 1]. (4.1) The followig coditios may be required i our theorems. Assumptio B. (B1) is positive o [, 1] with cotiuous derivative $; (B2) For ay $>, is positive o [$, 1&$] with cotiuous derivative $; (B3) There exists $ > <M 1, M 2 < such that (t)m 1 t, t #(,$ ] (t)m 2 (1&t), t #[1&$,1). I the ext lemma, we show that { give by (4.1) is secod-order Hadamard differetiable with the proof deferred to Sectio 6. Lemma 4.1. Let,=- let (G, H )#E with 1 dg <. The, (i) Uder Assumptio (B1), we have that the fuctioal { give by (4.1) is first-order Hadamard differetiable at (G, H ) with the first-order Hadamard derivative {$ (G, H )(G, H)=2 1 [G,$(H ) H+,(H ) G] G,(H ) dh + 1 G 2,2 (H ) dh; (4.2)

22 28 REN AND SEN (ii) If i additio to Assumptio (B1), we assume that has cotiuous secod derivative ", the the fuctioal { give by (4.1) is secodorder Hadamard differetiable at (G, H ) with the first-order Hadamard derivative give by (4.2), the secod-order Hadamard derivative {" (G, H )(G, H)=2 1 [G,$(H ) H+,(H ) G] 2 dh where G, H # D[, 1] [G,$(H ) H+,(H ) G] G,(H ) dh +2 1 [2,$(H ) GH+G,"(H ) H 2 ] G,(H ) dh, (4.3) I a special case of Lemma 4.1 with G #, weaker coditios are required o. We state this case as a corollary. From Corollary 3.4 Propositio 3.5, the proof is similar to that of Lemma 4.1. Corollary 4.2. Uder Assumptio (B1), for H # D[, 1] with 1 dh <, the fuctioal { give by (4.1) is secod-order Hadamard differetiable at (, H ) with the first-order Hadamard derivative { (, H )(G, H)#, G, H # D[, 1] (4.4) the secod-order Hadamard derivative {" (, H )(G, H)=2 1 (H ) G 2 dh, G, H # D[, 1]. (4.5) To study the asymptotic distributio of {(U *, U ) give by (1.4), we are particularly iterested i the followig special case for (U *, U ): Assumptio C. X 1, X 2,..., X are i.i.d. with a cotiuous d.f. F. I this case, we have U*= U =U i Assumptio (A1), because we always have : c i =, 1. (4.6)

23 SECOND ORDER HADAMARD DIFFERENTIABILITY 29 Theorem 4.3. Uder Assumptio (A), (B1) (C), the fuctioal { give by (4.1) is secod-order Hadamard differetiable at (, U) with derivatives where G, H # D[, 1]. Therefore, C 2 {(U *, U )=C 2 1 {$ (, U) (G, H)# (4.7) {" (, U) (G, H)=2 1 G 2 (U ) du, (4.8) C 2 {(U *, U ) w D 1 W 2 (t) (t) dt= : U* 2 (U) du+o p (1), as (4.9) j=1 * j Z 2 j, as, (4.1) where W is a Gaussia process o [, 1] with mea covariace #(s, t)=mi[s, t]&st, (4.11) Z j are idepedet stard ormal rom variables, * j are the eigevalues for the followig eigevalue problem: 1 #(s, t) - (s) - (t) /(t) dt=*/(s). (4.12) The proof of Theorem 4.3 is give i Sectio 6. It is clear that i Theorem 4.3, we require to be bouded. To hle more geeral weight fuctios, we first establish the followig lemma, the exted Theorem 4.3 to ubouded weight fuctios. The proofs are deferred to Sectio 6. Uder assumptio (C), we deote W (t)=c U*(t)= : where Y i =F(X i ) are i.i.d.r.v.'s with d.f. U. c i (I[Y i t]&t), t # [, 1], (4.13)

24 21 REN AND SEN Lemma 4.4. Uder Assumptio (A), (B2)(B3) (C), we have that for ay =>, there exists $> N such that for N P {} $ P {} 1 1&$ P {} $ P {} 1 W 2 (U ) du } = 1&= (4.14) = W 2(U ) du } = 1&= (4.15) = 1&$ W 2 (U) du } = 1&= (4.16) = W 2 (U) du } = 1&=. (4.17) = Theorem 4.5. Uder Assumptio (A), (B2)(B3) (C), the fuctioal { give by (4.1) satisfies C 2 {(U *, U )=C 2 1 C 2 {(U *, U ) w D 1 U* 2 (U) du+o p(1), as, (4.18) W 2 (t) (t) dt= : j=1 * j Z 2 j, as, (4.19) where W, Z j * j are as those i Theorem 4.3. Remark 4. Note that our coditio (B2) is required for havig the decompositio of W(t) - (t) i (4.19) (see Aderso Darlig, 1952). 5. APPLICATIONS I this sectio, we give the specific limitig distributios of those test statistics i Examples 14 6 of Sectios 1 2. Example 1. I model (1.2), we assume that the error variables = i 's are cotiuous. The, we kow that uder H, X 1,..., X are i.i.d.r.v.'s with some cotiuous d.f. F. Assumig (A2)(A3), from Theorem 4.5, we kow that the test statistics 9(S*, S ) has the followig limitig distributio uder H, C 2 9(S *, S )=C 2 {(U *, U ) w D 1 W 2 (t) (t) dt = : j=1 * j Z 2 j, as, (5.1)

25 SECOND ORDER HADAMARD DIFFERENTIABILITY 211 where W, Z j * j are as those i Theorem 4.3, satisfies Assumptio (B2)(B3), { is give by (4.1). Particularly, for some special weight fuctio, the values of * j i (5.1) have bee give. If #1, the * j =1( j?) 2, j=1, 2,... (5.2) (Shorack Weller, 1986, p. 214). If (t)=[t(1&t)] &1,the (Aderso Darlig, 1952). * j =1[ j( j+1)], j=1, 2,... (5.3) Example 2. If the error distributio is cotiuous i model (2.1), we have that X 1,..., X are i.i.d.r.v.'s with a cotiuous d.f. F uder H. If for [c i ] [d i ], (A2)(A3) hold with \ = c id i \, as, the by Theorem 4.5, we have that uder H, C 2 {(U *, U )+D 2 {(V *, U ) =C 2 1 U* 2 (U) du+d2 1 as, V* 2 (U) du+o p(1), where V *=T * b F &1. Furthermore, from a geeralizatio of Theorem of Shorack Weller (1986, p. 93) from the proof of our Theorem 4.3 give i Sectio 6, we have C 2 1 D 2 1 U* 2 (U) du= 1 V* 2 (U) du= 1 W 2 1 (t) (t) dt+o p(1), as, W 2 2 (t) (t) dt+o p(1), as, where W 1 W 2 are Browia bridges with Cov[W 1 (s), W 2 (t)]= \[mi[s, t]&st]. Hece, we have that uder H, & (C S *(x), D T *(x))(c S *(x), D T *(x)) T (S (x)) ds (x) =C 2 9(S *, S )+D 2 9(T *, S )=C 2 {(U *, U )+D 2 {(V *, U ) w D 1 = : j=1 [W 2 (t)+w (t)] (t) dt * j (Z 2+Z 2 j j ), as, (5.4)

26 212 REN AND SEN max 2 {C [S *(x)] 2 (S (x)) ds (x), & D 2 [T *(x)] 2 (S (x)) ds (x) = & =max[c 2 9(S *, S ), D 2 9(T *, S )] w D max { : * j Z 2, : 2 j * j Z j, as, (5.5) = j=1 j=1 where * j are give by (4.12), (Z j, Z j) are i.i.d. bivariate ormal r.v.'s with E[Z j ]=E[Z j]=, Var[Z j ]=Var[Z j]=1 Cov[Z j, Z j]=\, satisfies Assumptio (B2)(B3), { is give by (4.1). For the special weight fuctio #1 (t)=[t(1&t)] &1, the values of * j are give by (5.2) (5.3), respectively. To fid the critical values of the tests usig (5.4) or (5.5), oe may geerate i.i.d. stard ormal r.v.'s: Y 1, Y 1,..., Y N, Y N for some large N. The, set Z j =Y j Z j=\ Y j &- 1&\ 2 Y j, j=1,..., N. Sice j=1 * j (Z 2 +Z 2 j j )rn * j=1 j(z 2 +Z 2 ) i (5.4) j j max[ * j=1 jz 2, j * 2 j=1 jz ]r j max[ N * j=1 jz 2, j N * 2 j=1 jz j ] i (5.5), the critical values of the test statistics (2.4) (2.5) ca be determied by the quatiles of N * j=1 j(z 2 +Z 2 ) j j max[ N * j=1 jz 2, j N * 2 j=1 jz j ], respectively. If a geeral liear model is cosidered, X i =:+c T i ;+= i,,...,, (5.6) where c i are kow p-vectors of regressio costats, ; is the p-vector ukow regressio parameters, = i are i.i.d.r.v.'s with a cotiuous d.f., we let D =(c 1,..., c ) T, D = &1 1 1 T D C =Diag(&d 1 & E,..., &d & E ), c i =C &1 (c i &c ) 7 = : c i c T, i where d i is the colum vector of D &D, c is a colum of D, &}& E sts for Euclidea orm. Also, let S *(x)=c &1 : c i I[X i x] (5.7)

27 SECOND ORDER HADAMARD DIFFERENTIABILITY 213 with lim max 1i &c i & 2 =, lim E 7 =7 &1 &1 T C & 2 E M<. The, uder H : ;= (vs. H 1 : ;{), we have that the test statistics & (C, S *) T (C S *) (S ) ds w D : max 1kp{ & j=1 (e T k C S *) 2 (S ) ds = * j Z T j Z j, as, (5.8) w D max 1kp{ : * j (e T Z k j) j=1 2=, as, (5.9) where e k is a p_1 vector with the kth compoet 1 others, * j are give by (4.12), Z j are i.i.d. N p (, 7) satisfies Assumptio (B2)(B3). The critical values of the tests ca be obtaied similarly as that for the case of p=2 outlied above. Example 3. Cosider the test statistic give i (2.9) for the 3-sample problem (2.6) with o-equal sample size. For [c i ] [d i ], we have \ = : c i d i =a 1 b 1 +a 2 b 2 +a 3 b 3 = (A2)(A3) hold if j, as, j=1, 2, 3. Hece, if (B2)(B3) hold for the weight fuctio, from (5.4) the test statistic (2.9) has the followig limitig distributio uder H, &{ 1\ F 1 & +1 2 S + + 2\ G 2 & +1 2 S + + 3\ H 3 & +1 2 S + = (S ) ds w D : j=1 * j / 2 j, as, (5.1) where / 2 j are i.i.d. Chi-square rom variables with degrees of freedom 2, for the special weight fuctio #1 (t)=[t(1&t)] &1, the values of * j are give by (5.2) (5.3), respectively.

28 214 REN AND SEN Example 4. Case (b): Cosider the test statistic give i (2.14) for the 3-sample problem (2.6). For [c i ], [d i ] [e i ] i (2.13), we have \ (c, d)= : k=1-1 2 c k d k =& - (& 1 )(& 2 ) \ (c, e)=& - (& 1 )(& 3 ), \ (d, e)=& - (& 2 )(& 3 ). (5.11) Suppose that j (1& j ), as, j=1, 2, 3, (\ (c, d), \ (c, e), \ (d, e)) coverges to (\ cd, \ ce, \ de ), as, that (B2)(B3) hold for the weight fuctio. The, (A2)(A3) hold, from (5.9), the test statistic (2.14) has the followig limitig distributio uder H, max { 1 & 1 &\ F 1 & +1 2 & 2 &\ G 2 & +1 3 & 3 &\ H 3 & +1 w D where Z j are i.i.d. N 3 (, 7) with max 1k3{ : * j (e T Z k j) j=1 2 S + (S ) ds, 2 S + (S ) ds, 2 S + (S ) ds =, as, (5.12) 2= 1 \ cd \ ce 7=_\ cd 1 \ de&, (5.13) \ ce \ de 1 for the special weight fuctio #1 (t)=[t(1&t)] &1, the values of * j are give by (5.2) (5.3), respectively. I particular, for Case (a) with equal sample size as described i Sectio 2, we have \ cd = \ ce =\ de =&12 i (5.13). Case (d): Cosider the test statistics give by (2.18) (2.18a) for the 3-sample problem (2.6). For [c i ], [d i ] [e i ] i (2.17), we have \$ (c, d)=\ (c, e), \$ (c, e)=\ (d, e), \$ (d, e)=\ (c, d), (5.14) where \ (c, d), \ (c, e) \ (d, e) are give i (5.11). Suppose that j, as, j=1, 2, 3, (\$ (c, d), \$ (c, e), \$ (d, e)) coverges to (\$ cd, \$ ce, \$ de ), as, that (B2)(B3) hold for the weight

29 SECOND ORDER HADAMARD DIFFERENTIABILITY 215 fuctio. The, (A2)(A3) hold, from (5.9) (5.8), the test statistics give by (2.18) (2.18a) have the followig limitig distributios uder H, max { (F 1 &G 2 ) 2 (S ) ds, & (G 2 &H 3 ) 2 (S ) ds, & (H 3 &F 1 ) 2 (S ) ds & = w D max 1k3{ : * j (e TZ k j), as (5.15) 2= j=1 &{ (F 1 &G 2 ) (G 2 &H 2 ) (H 3 &F 1 ) 2= (S ) ds w D : j=1 * j Z T j Z j, as, (5.16) respectively, where for 7$ give by (5.13) with \ cd, \ ce, \ de replaced by \$ cd, \$ ce, \$ de, respectively, Z j are i.i.d. N 3 (, 7$), for the special weight fuctio #1 (t)=[t(1&t)] &1, the values of * j are give by (5.2) (5.3), respectively. I particular, for Case (c) with equal sample size as described i Sectio 2, we have \$ cd =\$ ce =\$ de =&12, thus 7=7$. Remark 5. It is worth metioig that with equal sample size i the 3-sample problem (2.6), the limitig distributio of the test statistic give by (2.18a) is 3 2 * j=1 j/ 2, where j /2 j are i.i.d. Chi-square rom variables with degrees of freedom 2, because the eigevalues of 7$ i (5.16) are, Also, oe may ote that by usig Theorem 3.6 we ca easily show that uder a fixed alterative hypothesis, the limitig distributio of the test statistic give by (2.18a) is ormal. Remark 6. Oe may ote that i Examples 12, it is show that the SOHD method allows us to geeralize the Crame rvo Mises type of test statistics (Ha jek S8 ida k, 1967, p. 13) from model (1.2) to (2.1) i a rather straightforward way allows us to derive their asymptotic distributios coveietly. The ivestigatio o the power of the proposed tests here i compariso with the alterative tests, say, (ormal) rak tests,

30 216 REN AND SEN is techical, which will ot be studied i this curret paper. I Ha jek S8 ida k (1967, pp ), a compariso of the asymptotic local power of the oe-sided KolmogorovSmirov test with that of the ormal rak test was give. We may expect similar results for our geeralized Crame rvo Mises tests. Oe may also ote that for the 3-sample problem (2.6), the ull limitig distributios of the test statistic studied by Scholz Stephes (1987) the alterative test statistics costructed i Example 4 are just the special cases of that i Example 2 for liear regressio model. Example 6. Cosider the test statistic give by (2.27) for the idepedece test (2.24), let x=(u, v), y=(s, t), F 1 (u)=f(u, ) F 2 (v)=f(, v). It ca be show that uder H, - ms*(f &1 1 (u), F &1 2 (v))=- mu *(u, v) weakly coverges to a cetered Gaussia process G(x) with covariace fuctio #(x, y)=(mi[u, s]&us)(mi[v, t]&vt), S (F &1 1 (u), F &1 2 (v))=u (u, v) uiformly coverges to uv with probability 1. Followig the cocepts i Re Se (1995) for the first-order Hadamard derivative with bivariate rom vectors, it is easy to geeralize Theorems to their bivariate versios uder suitable coditios. Thus, we have that uder H, C 2 9(S *, S )=C 2 1 w D [U *(u, v)] 2 (U (u, v)) du (u, v) G 2 (u, v) (uv) du dv, as m. 6. PROOFS Proof of Propositio 3.1. Sice T Q are secod-order Hadamard differetiable at F T(F), respectively, for ay compact set 1 V of V compact set 1 W of W, we have T(F+tH)&T(F)&T$ F (th)& 1 lim T" 2 F(tH) = (6.1) t t 2 uiformly for ay H # 1 V, Q(T(F)+tG)&Q(T(F))&Q$ lim T(F) (tg)& 1 Q" 2 T(F)(tG) =. (6.2) t t 2

31 SECOND ORDER HADAMARD DIFFERENTIABILITY 217 uiformly for ay G # 1 W. By Propositio of Ferholz (1983), we kow {$ F =Q$ T(F) b T$ F, obviously {$ F # L 1 (V, Z). It is also obvious that {" F give by (3.9) is a elemet of L 2 (V, Z). From (6.1) we have T(F+tH)=T(F)+T$ F (th)+ 1 2 T" F(tH)+o(1) t 2, where o(1) coverges to uiformly for ay H # 1 V,ast. Hece, by the liearity of Q$ T(F), we have {(F+tH)&{(F)&{$ F (th) =Q(T(F+tH))&Q(T(F))&Q$ T(F) (T$ F (th)) =Q(T(F)+T$ F (th)+ 1 2 T" F(tH)+o(1) t 2 ) &Q(T(F))&Q$ T(F) (T$ F (th)) =Q(T(F)+t[T$ F (H)+ 1 2 tt" F(H)+o(1) t])&q(t(f)) It is easy to show that &Q$ T(F) (t[t$ F (H)+ 1 2 tt" F(H)+o(1) t]) t2 Q$ T(F) (T" F (H))+t 2 Q$ T(F) (o(1)). (6.3) 1$ W =[T$ F (H)+ 1 2 tt" F(H)+o(1) t; H # 1 V, t #[&1,1]] =[T$ F (H)+}(H, t); H # 1 V, t #[&1,1]] is compact, where }(H, t)=[t(f+th)&t(f)&t$ F (th)]t coverges to uiformly for ay H # 1 V,ast. Hece, by (6.2), we have 1 lim t t [[Q(T(F)+t[T$ F(H)+ 1 2tT" 2 F (H)+o(1) t])&q(t(f)) &Q$ T(F) (t[t$ F (H)+ 1tT" 2 F(H)+o(1) t])]& 1Q" 2 T(F)(T$ F (th))] Q" = lim T(F) (t[t$ F (H)+ 1 tt" 2 F(H)+o(1) t])&q" T(F) (T$ F (th)) t 2t 2 1 = lim [Q" 2 T(F)(T$ F (H)+ 1tT" 2 F(H)+o(1) t)&q" T(F) (T$ F (H))] t 1 = lim [Q" 2 T(F)(T$ F (H)+}(H, t))&q" T(F) (T$ F (H))] (6.4) t

32 218 REN AND SEN uiformly for ay H # 1 V.Let,(H, t)=q" T(F) (T$ F (H)+}(H, t))&q" T(F) (T$ F (H)); the,: V_[&1, 1] W is cotiuous,(h, t) coverges to for ay fixed H # 1 V,ast. Hece, for ay ope set O of W such that # O for ay H # 1 V, there exists a ope set O (H,)=O H _I (H,) of V_[&1, 1] such that O H is a ope set of V H # O H, I (H,) is a ope iterval # I (H,), [,(H, t)&,(h,)]#o if (H, t)# O (H,). Sice [O H ; H # 1 V ] is a ope coverig of 1 V sice 1 V is compact, there exists a fiite ope coverig of 1 V, say, 1 V / N O H i. Therefore, for ay H # 1 V ay small eough t, there exists i such that (H, t)#o (Hi,). Hece,,(H, t)=[,(h, t)&,(h i,)]#o. Therefore, we have 1 lim 2[Q" T(F) (T$ F (H)+}(H, t))&q" T(F) (T$ F (H))]= (6.5) t uiformly for ay H # 1 V. Sice Q$ T(F) is cotiuous, the lim t t 2 Q$ T(F) (o(1)) = lim Q$ t 2 T(F) (o(1))= (6.6) t uiformly for ay H # 1 V. Therefore, (6.3) through (6.6) imply that {(F+tH)&{(F)&{$ F (th)& 1 lim [Q" 2 T(F)(T$ F (th))+q$ T(F) (T" F (th))] = t t 2 uiformly for ay H # 1 V. Proof of Propositio 3.2. show that K For ay compact set 1 of D[, 1], we eed to uiformly for H # 1, ast, where " Rem 2(tH) t 2 "L p (6.7) Rem 2 (th)=l b (Q+tH)&L b Q&(L$ b Q) th& 1 2 (L" b Q) t2 H 2.

33 SECOND ORDER HADAMARD DIFFERENTIABILITY 219 Sice 1 is a compact set, for arbitrary =>, we ca choose H 1,..., H # 1 such that for ay H # 1, Sice for a give H i, if &H&H i &<=. (6.8) 1i L(Q(x)+tH i (x))&l(q(x))&l$(q(x)) th i (x) Rem 2 (th i )(x) & 1 = L"(Q(x)) 2 t2 H 2 (x) i t 2 t 2 = L$(!)&L$(Q(x)) t H i (x)& 1 2 L"(Q(x)) H 2 i (x), where! is betwee Q(x) Q(x)+tH i (x), by Lemma of Ferholz (1983), we have that } L$(!)&L$(Q(x)) t } M H i(x), where M is a boud for L". Therefore, for each i, } Rem 2(tH i )(x) t } M 2 1 H i (x) 2, where M 1 is a costat. Moreover, for x such that Q(x) A, Rem 2 (th i )(x) t 2, as t. So, by the Domiated Covergece Theorem, we have For ay H # 1, " Rem 2(tH i ) t 2 "L p, as t. (6.9) " Rem 2(tH) 2(tH i ) t 2 "L "Rem + 2(tH) p t 2 "L "Rem & Rem 2(tH i ) (6.1) p t 2 t 2 "L p

34 22 REN AND SEN Rem 2 (th)&rem 2 (th i ) t 2 = L(Q(x)+tH(x))&L(Q(x)+tH i(x)) t 2 & L$(Q(x))[H(x)&H i(x)] & 1 t 2 L"(Q(x))[H 2 (x)&h 2 (x)] i = L$(')&L$(Q(x)) t [H(x)&H i (x)]& 1 2 L"(Q(x))[H 2 (x)&h 2 i (x)], where ' is betwee Q(x)+tH(x) Q(x)+tH i (x). As above, by Lemma of Ferholz (1983), we have Hece, by (6.8), we have } L$(')&L$(Q(x)) t } M H(x)&H i(x). if 1i" Rem 2(tH) & Rem 2(tH i ) M t 2 t 2 1 =, (6.11) "L p where M 1 is a costat which depeds o 1. Therefore, (6.7) follows from (6.9) through (6.11). K Proof of Propositio 3.3. It suffices to show that lim t #(G +tg, H +th )&#(G, H ) &#$ (G, H )(tg, th )& 1 2 #" (G, H )(tg, th ) t 2 =, for ay G G, H H, as, where G, G, H, H # D[, 1]. Note that 1 t [#(G +tg 2, H +th )&#(G, H ) &#$ (G, H )(tg, th )& 1#" 2 (G, H )(tg, th )] = G [,(H +th )&,(H )] &,$(H ) G H t +G,(H +th )&,(H )&,$(H ) th & 1 2,"(H ) t 2 H 2 t 2 =[,$(!)&,(H )] G H [,"(')&,"(H )] G H 2, (6.12)

35 SECOND ORDER HADAMARD DIFFERENTIABILITY 221 where! ' are betwee H H +th. Sice," is cotiuous, the proof follows from (6.12). K Proof of Propositio 3.5. It suffices to show that lim Rem 2 (t G, t H ; #) =, t 2 for ay G G, H H, t, as, where G, G, H, H # D[, 1] such that (G +t G, H +t H )#E. Note that Hece, #(G +t G, H +t H )&#(G, H ) &t #$ (G, H )(G, H )& 1 2 t2#" (G, H )(G, H )=t 3 1 G 2 (x) dh (t). Rem 2 (t G, t H ; #) =t t 2 1 G 2 (x) dh (t). The proof follows from the oe of Lemma 3 by Gill (1989). K Proof of Theorem 3.6. The proof of (3.11) is similar to that of Theorem 3.1 by Re Se (1991). We sketch the idea of the proof as below. Let U * U be the cotiuous versio of U * U, respectively, with &U *&U *&C &1 max c i, a.s. (6.13) 1i &U &U &(+1) &1, a.s. (6.14) Sice C [U *&E[U *]] - [U &E[U ]] weakly coverge o (D[, 1], &}&) (see Shorack Weller, 1986, p. 19), by (A1), we have that W =C [U *&U*] V =- [U &U ] also weakly coverge o (D[, 1], &}&). If we deote Z =C (U *&U*, U &U ), we easily see that Z # C[, 1]_C[, 1]. Note that Z =C (U *&U *, U &U )+C (U *&U*, U &U ). (6.15)

36 222 REN AND SEN Hece, from (A2) (6.13)(6.15), we kow that C [U *&U*] C [U &U ] are relatively compact o C[, 1]. Sice, C[, 1] is complete separable, by Prohorov's Theorem (Billigsley, 1968), we have that for every =>, there exist compact sets K 1 K 2 i C(, 1] such that P[Z # K]>1&=, all 1, (6.16) where K=K 1 _K 2 is a compact set i C[, 1]_C[, 1]. For Q(G, H, t)= Rem(tG, th; {)t with G, H # D[, 1], the rest of the proof follows alog the lies of the proof of Theorem 3.1 by Re Se (1991), hece is omitted. The proof of (3.13) follows similarly by usig Q(G, H, t)=rem 2 (tg, th; {)t 2. K Proof of Lemma 4.1. We will oly prove (ii) sice the proof of (i) is quite similar. Note that the fuctioal { ca be expressed as a compositio of the followig secod-order Hadamard differetiable trasformatios: # 1 : D[, 1]_D[, 1] D[, 1]_D[, 1] defied by # 1 (G, H)= (G,(H), H), is, by Propositio 3.3, secod-order Hadamard differetiable at (G, H ) with derivatives #$ 1(G, H ) (G, H)=(G,$(H ) H+,(H ) G, H), #" 1(G, H ) (G, H)=(2,$(H ) GH+G,"(H ) H 2,), where G, H # D[, 1]; # 2 : D[, 1]_D[, 1] R defied by # 2 (G, H)= 1 G2 dh, is, by Propositio 3.5, secod-order Hadamard differetiable at (G,(H ), H ) with derivatives #$ (G, H)=2 2(G,(H ), H ) 1 G,(H ) GdH + 1 G 2,2 (H ) dh, #" (G, H)=2 2(G,(H ), H ) 1 G 2 dh +4 1 G,(H ) GdH, where G, H # D[, 1].

37 SECOND ORDER HADAMARD DIFFERENTIABILITY 223 We have {=# 2 b # 1. Hece, by Propositio 3.1. { is secod-order Hadamard differetiable at (G, H ) with derivatives give by (4.2) (4.3). K Proof of Theorem 4.3. From Corollary 4.2 Theorem 3.6, we immediately have (4.7)(4.9). For W give by (4.13), we have because we always have (4.6) E[W (t)]=, E[W (s) W (t)]=mi[s, t]&st, : c 2 i =1, 1. (6.17) For the special costructio (Shorack Weller, 1986, p. 93), by Theorem of Shorack Weller (1986, p. 14), we have } 1 W 2 (U) du& 1 W 2 (U) du } 1 W 2 &W 2 (U) du &(W 2 &W 2 )(U(1&U)) 14 & 1 (U)(U(1&U)) 14 du &(W &W)(U(1&U)) 14 & [&W &W&+2 &W&] _ 1 (U)(U(1&U)) 14 du =o p (1)(o p (1)+O p (1)) 1 (U)(U(1&U)) 14 du=o p (1), (6.18) where W is a Browia bridge. Sice for the special costructio W give i (4.13), the distributios of 1 W 2 (U) du are the same, we have that C 2{(U *, D ) w D 1 W 2 (t) (t) dt, as. (6.19)

38 224 REN AND SEN For the decompositio of 1 W 2 (t) (t) dt i (4.1), oe may see Aderso Darlig (1952) or Shorack Weller (1986, Chap. 5). K Proof of Lemma 4.4. Recall that we have U (t)=(+1) &1 : I[Y i t], where Y i 's are as those i (4.13). We otice that for the special costructio (Shorack Weller, 1986, p. 93) for our W, U, the distributios of $ W 2 (U ) du, 1 W 2 (U 1&$ ) du, $ W 2 (U) du 1 W 2 1&$ (U) du are the same, respectively. Hece, it suffices to establish (4.14)(4.17) for the special costructio, which will be also deoted by W U. Let Y (1),..., Y () be the order statistics of Y 1,..., Y. Note that for ay $>, $ W 2 (U ) du M 1 $ w 2 U du (+1) &1 : =(+1) &1 : =M 1 (+1) &1 : =M 1 (+1) &1 : W 2 (Y (i)) i(+1) I[Y (i)$] +(+1) &1 : W 2 (Y (i)) U (Y (i) ) I[Y (i)$] W 2 (Y (i)) i(+1) I[Y (i)$], (6.2) W 2 (Y (i))&w 2 (Y (i) ) I[Y i(+1) (i) $] W 2 (Y (i) ) i(+1) I[Y (i)$], (6.21) where W is a Browia bridge. From Theorem of Shorack Weller (1986, p. 14), we kow that &(W 2 &W 2 )(U(1&U)) 14 & &(W &W)(U(1&U)) 14 & [&W &W&+2 &W&] =o p (1)(o p (1)+O p (1))=o p (1). (6.22)