More higher-order eciency: Concentration probability. Yutaka KANO. Osaka University. (Final version, May 1998)

Transkript

1 More higher-orer eciecy: Cocetratio probability Yutaka KANO Osaka Uiversity (Fial versio, May 998) Ruig Title: MORE HIGHER-ORDER EFFICIENCY. Mailig Aress: Faculty of Huma Scieces, Osaka Uiversity, Suita, Osaka , JAPAN. Abstract Base o cocetratio probability of estimators about a true parameter, thir-orer asymptotic eciecy of the rst-orer bias-ajuste MLE withi the class of rst-orer bias-ajuste estimators has bee well establishe i a variety of probability moels. I this paper we cosier the class of seco-orer bias-ajuste Fisher cosistet estimators of a structural parameter vector o the basis of a i.i.. sample raw from a curve expoetial-type istributio, a stuy the asymptotic cocetratio probability, about a true parameter vector, of these estimators up to the fth-orer. I particular, (i) we show that thir-orer eciet estimators are always fourth-orer eciet; (ii) a ecessary a suciet coitio for fth-orer eciecy is provie; a ally (iii) the MLE is show to be fth-orer eciet. Key Wors a Phrases: Bias-ajustmet, curve expoetial istributios, Egeworth expasio, maximum likelihoo estimator, Fisher-cosistecy. AMS 99 subject classicatios: 62F2 {{

2 Itrouctio I the recetly publishe book by J. K. Ghosh (994), he metios, i Sectio 6. locate after escriptio of thir-orer (asymptotic) eciecy, that Where o we go from here: Give that rst-orer eciecy implies seco-orer eciecy, it is atural to cojecture that thir-orer eciecy implies fourth-orer eciecy. The proof of that must be very messy. Oe may also ask if aythig like thir-orer eciecy hols for b whe we go to the fth-orer. Ghosh a Siha (982) show with a couterexample that this is ot possible. With fairly simple proofs, this paper gives a armative aswer to the cojecture above a establishes the fth-orer eciecy of the MLE, uer a slightly ieret biasajustmet. Let (X ; A; P ) be a complete a separable probability space for each 2, where, a ope omai of p, is a parameter space of. Let x be a p-imesioal raom vector (X! p, A-measurable) with a cotiuous expoetial-type istributio: exp( x ())(x); (.) where () is a carrier measure o p (see, e.g., Amari, 985, sectio 4.) a the prime eotes the traspose of a vector or a matrix. The real-value fuctio, (), o is ee as Z () = log p exp( x)(x): Assume that the parameter p-vector is a fuctio of a more basic parameter q-vector u (p > q), i.e., = (u) (.2) is a fuctio from, a ope omai of q, to ( p ). The family of the istributios (.) with the structure = (u) is sai to be of curve expoetial-type. Let x ;... ; x be ietically a iepeetly istribute (i.i..) raom p-vectors ee o the probability space (X ; A ; P ), each of which is istribute accorig to (.) with the structure (.2). I this paper, we shall stuy asymptotic properties of estimators for the ukow parameter vector u o the basis of the raom sample x ;... ; x. The likelihoo fuctio L(u), ee o, is writte as L(u) = Y j= exp((u) x j ((u))) = expf((u) x ((u)))g; {2{

3 where x = P j= x j. Note that x is a suciet statistic for (a hece for u). The maximum likelihoo estimator (MLE), ^u = ^u(x), is ee as a solutio to the maximizatio problem: Some assumptios o the MLE will be state later. max L(u): (.3) u 2 Let g() be a fuctio from p to q, smooth i a eighborhoo of a arbitrarily xe (u)(= E[x]) (see (3.3)). The fuctio g(x) is sai to be a Fisher cosistet estimator if a oly if g((u)) = u for all possible u 2. The property exclues Hoges' pathological example (see Le Cam, 953). Let g (x) a g(x) be Fisher cosistet estimators of u, a let g 3 (x) a g3 (x) be correspoig bias-ajuste estimators which have the same asymptotic bias up to a suitable orer. Cosier the followig statemet: For a positive iteger k, P (u) f p (g 3 (x) u) 2 Cg P (u) f p (g 3 (x) u) 2 Cg + o((k)=2 ); (.4) where P (u) eotes the probability w.r.t. (.) with (.2) a C is ay Borel covex set of q symmetric about the origi. The statemet (.4) asserts that the estimator g 3 (x) is more cocetrate about the true parameter u tha g 3 (x) up to the orer o( (k)=2 ). If g 3 (x) satises the relatio (.4) for ay Fisher cosistet estimator g 3 (x) at k =, the estimator g (x) is sai to be rst-orer (asymptotically) eciet. A rst-orer eciet estimator g (x) is sai to be seco-orer eciet i (.4) hols true at k = 2 for ay rst-orer eciet estimator g(x); i geeral, (K )th-orer eciet estimator g (x) is sai to be Kth-orer eciet i (.4) hols true at k = K for ay (K )th-orer eciet estimator g(x) (see, e.g., Akahira a Takeuchi, 98, page 8; Amari, 985, page 3). The purpose of this paper is to give a almost complete aswer to some atural questios o the fourth- a fth-orer eciecy of the MLE. We show that uer a cer- which is of orer O p tai seco-orer bias-ajustmet, the fourth-orer term of the cocetratio probability,, of thir-orer eciet estimators oes ot epe o the estimators, that is, thir-orer eciecy automatically implies fourth-orer eciecy. The cosequece correspos to the impressive result, prove by Pfazagl (976), that rstorer eciecy implies seco-orer eciecy. Next, a ecessary a suciet coitio for fth-orer eciecy is give, a we apply the coitio to establish the fth-orer eciecy of the MLE. {3{

4 Techically, the followig two thigs are ew: (i) It is see that a alterative biasajustmet factor must be employe, otherwise Ghosh a Siha's (982) couterexample to the fth-orer eciecy of the MLE is alive; (ii) partial ieretial equatios (PDE) are erive to evaluate characteristic fuctios. The metho of evaluatio through the PDE is ew a successfully reuces the computatio, a eve though the metho is applie to prove the thir-orer eciecy, it is ew. Rather tha emphasizig the ovelty of these techiques, we hope that oe will evelop these techical tools to establish socalle absolute asymptotic eciecy, i.e., the MLE is asymptotically eciet at ay orer, a cojecture by Rao, Siha a Subramayam (982, Remark 3.). A overview of the paper is as follows: Sectio 2 briey reviews the asymptotic eciecy, a Sectio 3 gives otatio use throughout the paper. Mai results are state i Sectio 4. A outlie of the proofs of the mai results is escribe i Sectio 5; some techical etails are presete i Appeix. 2 Brief historical review The otio of (asymptotic) eciecy i estimatio problem was rst iscusse by Fisher (925). The rst-orer eciecy of the MLE i the sese of (.4) was prove by Rao(962), Bahaur (964) a Kaufma (966) for the case where a geeral esity fuctio w.r.t. the Lebesgue measure is permitte. Several reemets of the rst-orer eciecy have bee mae by Hajek (97), Iagaki (97), a Pfazagl (973a) amog others. The seco-orer eciecy of the MLE was prove by Pfazagl (973b); a the, as ote i Itrouctio, he establishe that rst-orer eciecy implies seco-orer eciecy uer bias-ajustmets (see Pfazagl, 976). Extesios to the multi-parameter case or to ot i.i.. cases were mae by Akahira (975), Akahira a Takeuchi (976), Hosoya (979) a Taiguchi (983) amog others. Yoshia (992, 93) recetly mae a attempt to establish seco-orer eciecy i iusio processes. The thir-orer eciecy of the MLE was show by Takeuchi a Akahira (978), Pfazagl a Wefelmeyer (978) a Ghosh, Siha a Wiea (98). Extesios to more complicate moels were mae by Takeuchi a Morimue (985) a Taiguchi (986,87). Thorough theory of asymptotic eciecy up to the thir-orer i estimatio problems is escribe by the well-writte text books by Akahira a Takeuchi (98), Pfazagl (985), Taiguchi (99), a Ghosh (994). I the iscussio of seco- or {4{

5 thir-orer optimality i estimatio, the mai ieas are bias-ajustmets a Egeworth expasios. There are more tractable but somewhat iirect criteria of asymptotic eciecy such as the quaratic loss a loss of iformatio. Plety of research o seco- or thir-orer eciecy base o these criteria have bee oe by may authors icluig Rao (96,63), Ghosh a Subramayam (974), Efro (975), Eguchi (983), a Amari (985). Efro (975) a Hosoya (99) have poite out a certai equivalece betwee these criteria. There is little work o stuyig properties of 3 terms of loss fuctios, see Efro (975), Rao, Siha a Subramayam (982), a Ghosh a Siha (982). Note that the evaluatio of 3 terms of loss fuctios correspos to that of 2 terms of cocetratio probability. Kao (997) recetly prove that the MLE miimizes the 3 terms of a expasio of the quaratic loss of Fisher cosistet estimators. No work o over thirorer eciecy o the basis of cocetratio probability has bee yet mae; the preset work is the rst attempt i this irectio. 3 Notatio Let x = [x ;... ; x p ], a let A(x) = [a ij (x)] be a a 2a 2 -matrix value smooth fuctio. i h. Dee A(x) = A(x) i x x ; p a A(x) = x k k A(x) x. The higher-orer (matrix) erivatives are ee iuctively by <k> = x. For such matrix erivatives, see e.g., Betler a x Lee (978), Magus a Neuecker (985,88), a Kao (993,97). Let us simply write x x <k> with x <> = A <k> = k-fol z } { A A; k-fol right Kroecker (tesor) prouct of the same matrix A. Dee A <> =. The symmetric tesor for the Kroecker prouct of p-vectors a ;... ; a k is eote by N p <k>, which operates N p <k>(a a k ) = X a() a (k) =k!; where the summatio rus over all permutatios (();... ; (k)) of (;... ; k). See e.g. Sterberg (964, Sectio.3) for the symmetric tesor. Let g(x) be a q -value smooth fuctio ee o a eighborhoo of x 2 p. The Taylor series of g(x) about x is expressible i the form: g(x) = X k= G k <k> k! (x x ) <k> with G k = x {5{ g(x) x=x :

6 The matrix of the erivatives, G k, has a importat property: G k N p <k> = G k : (3.) Recall that x ;... ; x are a i.i.. raom sample istribute accorig to the istributio (.) with (.2). Dee 9 k` = 9 k`((u)) = <k> <`> () =(u) : (3.2) Sice the cumulat geeratig fuctio of the istributio is writte as log E (u) [e is xj ] = ((u) + is) ((u)), we have E (u) [x j ] = 9 ((u) (= (u); say); j = ;... ; (3.3) where E (u) [] eotes the expectatio w.r.t. the probability measure i (.) with the structure (.2). Note that 9 k` = E (u) [(x j (u)) <k> (x j (u)) <`> ] for k + ` = 2; 3. Put z = p (x (u)) a 9 k`() = z <k> z <`>. The E (u) [9 k`()] = 9 k` for k + ` = 2; E (u) [9 k`()] = p 9 k` for k + ` = 3; E (u) [9 4 ()] = 3N p <4>9 <2> 2 + O : See Holmquist (988) a Kao (993, 97) for more geeral results. 4 Mai results First of all, we state some assumptios. Let u be a arbitrarily xe (ier) poit of, a let U ( ) be a eighborhoo of u. We eote by k the k-imesioal Borel -el. (A) (u) is ( q \ ; p \ )-measurable a smooth i u 2 U. Deote 2 = 2 (u) = a (u) i (3.3). (u) for u 2 u U. Recall the eitios of 9 k`() i (3.2) {6{

7 (A2) rak(2 (u)) = q a rak(9 ((u))) = p at u = u. Remember that g(x) is a q -value fuctio from p. (A3) g(x) is ( p ; q )-measurable a smooth i a eighborhoo of (u ), a g(x) is Fisher cosistet, i.e., g((u)) = u for ay u 2 U. page 7). Assumptio (A3) is also sai to be locally stable (see Pfazagl a Wefelmeyer, 978, (A4) The optimizatio problem (.3) has a uique solutio, ^u(x), say, for ay x i a eighborhoo of (u). The MLE ^u(x) is ( p ; q )-measurable a cotiuous at x = (u) (u 2 U ). The cotiuity assumptio esures (strog) cosistecy of ^u(x). Assumptios o a u are eee to guaratee the cotiuity; we o ot iscuss it here. See Rao (973) a Kao (986) for etails. It follows from (A4) that for ay x i a eighborhoo of (u ), ^u(x) is a solutio to the equatio: u L(ujx) = 2 (u) fx (u)g = : (4.) The fuctio u L(ujx) is smooth i (x; u) i a eighborhoo of ((u ); u ), a u u L(ujx) = 2 (u )9 ((u ))2 (u ) (= i u ; say) (4.2) (x;u)=((u);u) is positive eite i view of (A2). The cotiuity i (A4) esures that (x; ^u(x)) stays i a eighborhoo of ((u ); u ) for ay x close to (u ). Thus, applicatio of the implicit fuctio theorem to (4.) shows that ^u(x) is smooth i a eighborhoo of (u ). Note that i u i (4.2) is the Fisher iformatio matrix. Bias ajustmets play a importat role i stuyig higher-orer asymptotic optimality of estimators. Recall z = p (x (u)) a write G k = G k ((u)) = x <k> g(x) x=(u) The fuctio G k (x) is ee merely o a eighborhoo of (u ), but we exte G k (x) to a fuctio ee o p which is measurable i x a smooth i a eighborhoo of (u ). {7{ :

8 The estimator g(x) is the expae formally as g(x) = u + p G z + 2 G 2z <2> + 6 p G 3z <3> + ; a hece the asymptotic bias is expressible i the form: E (u) [g(x) u] = 2 G G G 49 <2> 2 + O 3 : We shall make a rst-orer bias ajustmet (or correctio) via b (x) = 2 G 2(x)9 2 ((g(x))): (4.3) It shoul be ote that the ajustmet factor is slightly but essetially ieret from the oe employe i the literature: G ^ = 2 G 2((g(x)))9 2 ((g(x))). Either of the factors ca be employe to establish the seco- a thir-orer optimality of the MLE, i other wors, the optimality oes ot epe o the istributio of b (x) but oly o its coverget poit. I cotrast, the istributio oes iuece upo the terms which cotribute to more higher-orer optimality. Thus, the choice of the bias-ajustmet factor is essetial. bias: The rst-orer bias-ajuste estimator has the followig seco-orer asymptotic 8 < E (u) g(x) u b (x) = <2> 2 2 : x b (x) x=(u) 9 = ; G G 49 <2> 2 + O 3 = 2 b 2(u) + O 3 ; We make a seco-orer bias-ajustmet by b 2 (g(x)) (= b 2 (x), simply). The estimator after the seco-orer bias-ajustmet is eote by say: g 3 (x) = g(x) b (x) 2 b 2(x): (4.4a) I the same maer, the MLE ^u(x) is bias-ajuste up to the seco-orer: Let G k = Gk ((u)) = x <k> {8{ ^u(x) x=(u) :

9 The, ^u 3 (x) = ^u(x) b (x) 2 b 2 (x); (4.4b) where b (x) a b 2 (x) are are ee as b (x) a b 2 (x) with Gk for G k. As a result, it hols formally E (u) [g 3 (x) u] = O 3 ; E(u) [^u 3 (x) u] = O 3 : (4.5) It is kow that g(x) is rst-orer eciet i G = G (= i u 2 ), see e.g., Takeuchi a Akahira (978), formula (2.3); uer the rst-orer bias ajustmet, the rst-orer eciet estimator is always seco-orer eciet (Pfazagl, 976), a the estimator is thir-orer eciet i G 2 = G2. Although there woul ot be suitable refereces which explicitly state the last statemet, it coul be easily euce from the literature (see e.g. Akahira a Takeuchi, 98, pages 55-58). As a result, the bias-ajuste MLE is rst-, seco- a thir-orer eciet withi the class of bias-ajuste Fisher cosistet estimators. The ext theorem is the mai result of this paper. Theorem. Suppose that x... ; x are i.i.. p -value raom vectors ee o (X ; A ; P ), where P is of cotiuous curve expoetial-type (.) with the structure = (u) (u 2 U ). Write x = P j= x j. Assume (A){(A4). Let the MLE ^u(x) be ee by (.3) a let g(x) be thir-orer eciet; let ^u 3 (x) a g 3 (x) be seco-orer bias-ajuste i (4.4). Let C 2 q be a arbitrary covex set symmetric about the origi. The there exists a oegative quatity 2^u;g;C such that for each u 2 U, (i) (ii) P (u) f p (g 3 (x) u) 2 Cg uiformly i C a that = P (u) f p (^u 3 (x) u) 2 Cg 2 2^u;g;C + o 2 2^u;g;C = if a oly if G 3 = G 3, provie that C is of positive Lebesgue measure. The result (i) implies that the term of orer 3=2 of the LHS coicies with that of the RHS, that is, uer the seco-orer bias-ajustmet, the terms of orer 3=2 of expasios of the cocetratio probability of thir-orer eciet estimators are iepeet of the estimators; the terms of all such estimators are the same as the MLE's. As a result, we have the followig. {9{

10 Corollary. implies fourth-orer eciecy. Uer the seco-orer bias-ajustmet i (4.4), thir-orer eciecy It is see that oe ca ot istiguish thir-orer eciet estimators via Egeworth expasios up to the orer 3=2. The statemet (ii) of Theorem gives a ecessary a suciet coitio for fth-orer eciecy. We apply the coitio to get Corollary 2. eciet. Uer the seco-orer bias-ajustmet i (4.4), the MLE is fth-orer We shall e by statig a cosequece o asymptotic completeess. Let ^u 33 (x) = ^u(x) b (x) + b 2 2 (x) + b (x) + b 2(x) ; (4.6) 2 where b i (x) a b i (x) are the bias-ajustmet factors for g(x) a ^u(x), respectively. Theorem 2. Suppose that x... ; x are i.i.. p -value raom vectors ee o (X ; A ; P ), where P is of cotiuous curve expoetial-type (.) with the structure = (u) (u 2 U ). Write x = P j= x j. Assume (A){(A4). Let the MLE ^u(x) be ee by (.3), a let C 2 q be a arbitrary covex set symmetric about the origi. For ay Fisher cosistet thir-orer eciet estimator g(x), we ee ^u 33 (x) by (4.6). The, P (u) f p (g(x) u) 2 Cg = P (u) f p (^u 33 (x) u) 2 Cg 2 2^u;g;C + o 2 uiformly i C, where 2^u;g;C is ee i Theorem. Theorem 2 states that for ay thir-orer eciet estimator g(x), there exists a moie MLE ^u 33 (x) which is ot iferior to g(x) up to the orer 2 a that ^u 33 (x) oes have a higher cocetratio probability tha g(x) whe G 3 6= G3 a C is of positive Lebesgue measure. The result shows a ki of asymptotic completeess to the fourtha fth-orer, cf. Pfazagl a Wefelmeyer (978) a Akahira, Hirakawa a Takeuchi (988). 5 Proofs: Outlie {{

11 I this sectio we shall escribe a outlie of the proofs of the theorems give i Sectio 4. The formal Egeworth expasio of the istributio of p (g 3 (x) u) is obtaie as follows: For ay xe (u; t) 2 U 2 q, expa formally the characteristic fuctio of p (eg 3 (x)u) with eg 3 (x) a approximate statistic to g 3 (x) up to the orer 2 a make the iverse Fourier trasform of the expae characteristic fuctio, to get a expasio of the esity fuctio of p (g 3 (x) u). We x (u; t) 2 U 2 q throughout this sectio. Notice that x b (x) x=(u) = x 2 G 2(x)9 2 ((g(x))) x=(u) = 2 fg 3(I p 9 2 ) + G G g; a the the approximate statistic eg 3 (x) to g 3 (x) obtaie by expaig g 3 (x) up to the orer 5=2 ca be expresse i the form: eg 3 (x) = u + p G z + 2 G 2(z <2> 9 2 ) 2 p G G z + 6 p G 3fz <3> 3(I p 9 2 )z g + 2 P (z ); (5.) where P (z ) = P () (z ) + p P (2) (z ) = O p () is a ite polyomial of z. Sice x j has a cotiuous istributio, the istributio meets Cramer's coitio: lim sup jjsjj! E (u) [e is (xj(u)) ] < by the virtue of Bhattacharya a Deker (99, Lemma.5). Therefore, the so-calle formal Egeworth expasio of the istributio of p (g 3 (x) u) is vali up to the orer 2 i view of Theorem 2. of Bhattacharya a Deker (99) (see also Bhattacharya a Ghosh, 978). Accorigly, we evaluate asymptotic cocetratio probability of the approximate estimator eg 3 (x) about the true parameter u up to the orer 5=2 or of the staarize form p (eg 3 (x) u) about the zero vector up to the orer 2, rather tha that of g 3 (x) itself. Let eu 3 (x) be the approximate estimator costitute from the MLE ^u 3 (x) i the same maer as eg 3 (x). By (5.) we ca express the staarize forms of eg 3 (x) a eu 3 (x) as p (eg 3 (x) u) = G z + 2 p G 2(z <2> 9 2 ) 2 G G z + 6 G 3fz <3> 3(I p 9 2 )z g + p P (z ) {{

12 a p (eu 3 (x) u) = G z + 2 p G 2 (z <2> 9 2 ) G G z 2 + G 3 fz <3> 3(I p 9 2 )z g + p P (z ); 6 (5.2) where P (z ) is ee as P (z ) with G k for G k. We have otice that G = G a G 2 = G2 for g(x) to be thir-orer eciet. I this case, we have p (eg 3 (x) eu 3 (x)) = p (eg 3 (x) u) p (eu 3 (x) u) = 6 (G 3 G 3 )fz <3> 3(I p 9 2 )z g + p Q (z ); (5.3) where Q (z ) = P (z ) P (z ) is boue i probability a a ite polyomial of z (with coeciets smooth i u 2 U ) because so are P (z ) a P (z ). As euce from (4.5), the seco-orer bias-ajustmet leas to which is ot merely formal but vali. a E (u) [ p (eg 3 (x) eu 3 (x))] = O From (5.3) a (5.4), oe ca show E (u) [e itp (eu 3 (x)u) p (eg 3 (x) eu 3 (x))] = O 2p 2p ; (5.4) (5.5) E (u) [e itp 3 p (eu (x)u) (eg 3 (x) eu 3 (x)) p (eg 3 (x) eu 3 (x)) ] = 6 (G 2 3 G 3 )9 <3> (G 3 G 3 ) e t i u t=2 + O 2p : (5.6) The Appeix gives the ow of the proof of (5.5). The full proofs of these cosequeces a (5.9) are give i Kao (995). It follows that E (u) [e itp (eg 3 (x)u) ] =E (u) [e itp (eu 3 (x)u) e itp (eg 3 (x)eu 3 (x)) ] =E (u) [e itp (eu 3 (x)u) f + 2! (itp (eg 3 (x) eu 3 (x))) 2 + R g] + O 2p (by (5.5)) =E (u) [e itp (eu 3 (x)u) ] (it) (G 3 G 3 )9 <3> (G 3 G 3 ) (it) e t i u t=2 (5.7) {2{ + O 2p (by (5.6));

13 where R is the remier term of the expasio above, a we have use je (u) [e itp (eu 3 (x)u) R ]j E (u) [jr j] E (u) [jt p (eg 3 (x) eu 3 (x))j 3 =3!] (5.8) = O 3 i view of (5.3). As a result of (5.7), the ierece i the cocetratio probability betwee p (eg 3 (x) u) a p (eu 3 (x) u) base o the Egeworth expasios appears i the 2 terms, which result from the seco term of the RHS of (5.7). Thus, for each u 2 U, the probability that p (eg 3 (x) u) be i the covex regio C ca be expresse i the form: P (u) f p (g 3 (x) u) 2 Cg = P (u) f p (^u 3 (x) u) 2 Cg 2 2^u;g;C + o 2 uiformly i C, where 2^u;g;C = 2 vec(g 3 G 3 ) Z 9 <3> C fi u i u xx i u g N q (xj; i u )x vec(g 3 G 3 ): (5.9) The vec-operator vec(a) for a a 2 a 2 matrix A eotes a a a 2 -vector obtaie by stackig a 2 colum vectors of A i orer, see Heerso a Searle (98) a Magus a Neuecker (988) for its properties. The expressio (5.9) is obtaie by makig the iverse Fourier trasform of the seco term i (5.7). It is kow that the matrix of the itegrals Z C fi u i u xx i u g N q (xj; i u )x is positive eite, provie that C is of positive Lebesgue measure a covex, symmetric about the origi. See Pfazagl (985, Lemma 3.2.4) for a proof. Thus, 2^u;g;C is oegative, a 2^u;g;C = if a oly if G 3 = G 3. The proof of Theorem is complete. Proof of Theorem 2. ote that Theorem 2 ca be prove i the same maer. The reaer shoul p (g(x) ^u 33 (x)) = p (g 3 (x) ^u 3 (x)) (so that the correspoig approximate statistics are the same) a that the same erivatio i (5.7) hols eve though eu 3 (x) is replace with eu 33 (x) i e itp 3 (eu (x)u). Q.E.D. {3{

14 Appeix Here we give a coese proof of (5.5). See Kao (995) for the full etaile proof. Kao (997, Lemma 3.3) showe that for ay k 2, there exists a q 2 qp k matrixvalue smooth fuctio B k = B k (u) ee o U such that Gk = B k (2 I p k)n p <k> for ay u 2 U. The thir-orer eciecy, i.e., G 2 = G 2, imposes a restrictio o G 3 : (G 3 G 3 )(9 2 I p 2) = (G 3 G 3 )(I p 9 2 I p ) = (G 3 G3 )(I p ) = : I particular, (G 3 G 3 )9 <3> Let G 3 =. = (u; t) =E (u) [ p (eg 3 (x) eu 3 (x))e itp (eu 3 (x)u) ] a 2 = 2 (u; t) =E (u) [ p (eg 3 (x) eu 3 (x))e itp (eu 3 (x)u) z ]; where (u; t) 2 U 2 q. Dieretiatig i 's by t, we have the followig partial ieretial equatios (PDE): where i t = 8 (t i u ) i 9 + Ci (u; t) (i = ; 2); (A.) C (u; t) = p i u i u + i 2 p (B 2 vec(9 2 ) G ) 2 p (t 2 )B + i u B + 2 O 2p (A.2a) a C 2 (u; t) = i p 2 u (i u I p ) + i vec(9 G ) + O 2 : (A.2b) If C i (u; t)'s were regare as urelate to i 's, the PDE (A.) coul be solve i a close form: Let t k = [t ;... ; t k ; ;... ; ], C (u; t) = [c () {4{ (u; t);... ; c() (u; t)] a q

15 C 2 (u; t) = [C (2) (u; t);... ; C(2) q (u; t)]. The, usig E (u) [ p (eg 3 (x) eu 3 (x))] = O a E (u) [ p (eg 3 (x) eu 3 (x))z ] = O 2, we have a =e t i u t=2 (E (u) [ p (eg 3 (x) eu 3 (x))] + =e t i u t=2 qx k= Z tk qx k= c () k (u; t k)e t k i u tk=2 t k + O 2 =e t i u t=2 (E (u) [ p (eg 3 (x) eu 3 (x))z ] + =e t i u t=2 qx k= Z tk Z tk qx k= ) c () k (u; t k)e t k i u tk=2 t k 2p Z tk C (2) k (u; t k)e t k i u tk=2 t k + O 2 : Expa a 2 as = A + p A A 3 + O 2p ) C (2) k (u; t k)e t k i u tk=2 t k 2p (A.3a) (A.3b) a 2 = B + p B 2 + O 2. It suces to show A = A 2 = A 3 = for each (u; t). We rst ote that Z tk Z tk Z tk t k = O ; (t 2 )t k = O ; Z tk u t k = O ; (A.4a) 2 u t k = O (k = ;... ; q) uiformly i (u; t) i ay compact set of U 2 q. (k = ;... ; q) ui- R t It follows from (A.2a) a (A.4) that k C (u; t)t k = O p R formly i (u; t) i ay compact set of U 2 q t cotaiig (u ; ), so that k O p hece, ; which leas to = O Z tk t k = O p p ; (A.4b) c() k (u; t k)t k = i view of (A.3a). As a result, A equals a Z tk u t k = O p uiformly i (u; t) i ay compact set of U 2 q. Substitutio of (A.4b) a (A.5) ito (A.2b) shows R t k C 2(u; t)t k = O p set of U 2 q cotaiig (u ; ), so that R t k 2 = O p C(2) k i view of (A.3b). As a result, B equals. Thus, it is see that startig i = O p (A.5) (k = ;... ; q) uiformly i (u; t) i ay compact (u; t k)t k = O ; which leas to, we get A = B = a i = O p Repeatig this process, we have A 2 = B 2 = a i = O. The, we get 2 A3 = a = O. The proof is complete. 2p {5{.

16 Ackowlegmets. The author woul like to express his thaks to Professor M. Akahira of Uiversity of Tsukuba a Dr M. Taiguchi of Osaka Uiversity for ivaluable iscussio o higher-orer eciecy. REFERENCES [] Akahira, M. (975). A ote o the seco orer asymptotic eciecy of estimators i a autoregressive process. Rep. Uiv. Electro-Comm., 26, [2] Akahira, M., Hirakawa, F. a Takeuchi, K. (988). Seco a thir orer asymptotic completeess of the class of estimators. I Probability Theory a Mathematical Statistics, Lecture Notes i Mathematics, Vol.299 (S. Wataabe a Yu. V. Prokhorov, Es). pp Spriger-Verlag, Berli. [3] Akahira, M. a Takeuchi, K. (976). O the seco orer asymptotic eciecy of estimators i multiparameter cases. Rep. Uiv. Electro-Comm., 26, [4] Akahira, M. a Takeuchi, K. (98). Asymptotic Eciecy of Statistical Estimators: Cocepts a Higher Orer Asymptotic Eciecy, Lecture Notes i Statistics, Vol.7. Spriger-Verlag, New York. [5] Amari, S. (985). Dieretial-Geometrical Methos i Statistics, Lecture Notes i Statistics, Vol.28. Spriger-Verlag, Berli. [6] Bahaur, R. R. (964). O Fisher's bou for asymptotic variace. A. Math. Statist., 35, [7] Bhattacharya, R. a Deker, M. (99). Asymptotic Statistics. Birkhauser, Basel. [8] Bhattacharya, R. a Ghosh, J. K. (978). O the valiity of the formal Egeworth expasio. A. Statist., 6, [9] Betler, P. M. a Lee, Sik-Yum (978). Matrix erivatives with chai rule a rules for simple, Haamar, a Kroecker proucts. J. Math. Psychol., 7, [] Efro, B. (975). Deig the curvature of a statistical problem (with applicatios to seco orer eciecy). A. Statist., 3, [] Eguchi, S. (983). Seco orer eciecy of miimum cotrast estimators i a curve expoetial family. A. Statist.,, [2] Fisher, R. A. (925). Theory of statistical estimatio. Proc. Cambrige Philos. Soc., 22, [3] Ghosh, J. K. (994). Higher Orer Asymptotics. NSF-CBMS Regioal Coferece Series i Probability a Statistics, Vol.4. IMS, Haywar, CA. {6{

17 [4] Ghosh, J. K. a Siha, B. K. (982). Thir orer eciecy of the MLE A couterexample. Calcutta Statist. Assoc. Bull., 3, [5] Ghosh, J. K., Siha, B. K. a Wiea, H. S. (98). Seco orer eciecy of the MLE with respect to ay boue bowl-shape loss fuctio. A. Statist., 8, [6] Ghosh, J. K. a Subramayam, K. (974). Seco orer eciecy of maximum likelihoo estimates. Sakhya, Ser. A, 36, [7] Hajek, J. (97). A characterizatio of limitig istributios of regular estimators. Z. Wahr. verw. Geb., 4, [8] Heerso H. V. a Searle, S. R. (98). The vec-permutatio matrix, the vec operator a Kroecker proucts: A review. Liear a Multiliear Algebra, 9, [9] Holmquist, B. (988). Momets a cumulats of the multivariate ormal istributio. Stoch. Aal. App., 6, [2] Hosoya, Y. (979). Higher-orer eciecy i the estimatio of liear processes. A. Statist., 7, [2] Hosoya, Y. (99). Iformatio amout a higher-orer eciecy i estimatio. A. Ist. Statist. Math., 42, [22] Iagaki, N. (97). O the limitig istributio of a sequece of estimators with uiformity property. A. Ist. Statist. Math., 22, -3. [23] Kao, Y. (986). Coitios o cosistecy of estimators i covariace structure moel. J. Japa Statist. Soc., 6, [24] Kao, Y. (997). Beyo thir-orer eciecy. Sakhya, Ser. B, 59, [25] Kao, Y. (993). Beyo thir-orer eciecy: Supplemet. Techical Report, Uiversity of Osaka Prefecture. [26] Kao, Y. (995). More higher-orer eciecy: Cocetratio Probability. Techical Report of Statistical Group: DATA95-4, Tokyo, Japa. [27] Kaufma, S. (966). Asymptotic eciecy of the maximum likelihoo estimator. A. Ist. Statist. Math., 8, [28] Le Cam, L. (953). O some asymptotic properties of maximum likelihoo estimates a relate Bayes' estimates. Uiv. Califoria Publ. Statist.,, [29] Magus, J. R. a Neuecker, H. (985). Matrix ieretial calculus with applicatios to simple, Haamar, a Kroecker proucts. J. Math. Psychol., 29, {7{

18 [3] Magus, J. R. a Neuecker, H. (988). Matrix Dieretial Calculus with Applicatios i Statistics a Ecoometrics. Joh Wiley, Chichester. [3] Pfazagl, J. (973a). The Accuracy of the ormal approximatio for estimates of vector parameters. Z. Wahr. verw. Geb., 25, [32] Pfazagl, J. (973b). Asymptotic expasios relate to miimum cotrast estimators. A. Statist.,, [33] Pfazagl, J. (976). First orer eciecy implies seco orer eciecy. I Cotributio to Statistics (Hajek Memorial Volume), (J. Jureckova, E.), pp , Acaemia, Prague. [34] Pfazagl, J. (985). Asymptotic Expasios for Geeral Statistical Moels. Lecture Notes i Statistics, Vol.3. Spriger-Verlag, Berli. [35] Pfazagl, J. a Wefelmeyer, W. (978). A thir orer optimum property of the maximum likelihoo estimator. J. Multivariate Aal., 8, -29. [36] Rao, C. R. (96). Asymptotic eciecy a limitig iformatio. Proc. Fourth Berkeley Symp.,, [37] Rao, C. R. (962). Eciet estimates a optimum iferece proceures i large samples. J. Royal Statist. Soc., Ser. B, 24, [38] Rao, C. R. (963). Criteria of estimatio i large samples. Sakhya, Ser. A, 25, [39] Rao, C. R. (973). Liear Statistical Iferece a its Applicatios, 2 e. Wiley, New York. [4] Rao, C. R., Siha, B. K. a Subramayam, K. (982). Thir orer eciecy of the maximum likelihoo estimator i the multiomial istributio. Statist. Decisios,, -6. [4] Sterberg, S. (964). Lectures o Dieretial Geometry. Pretice-Hall Ic., Eglewoo Clis, NJ. [42] Takeuchi, K. a Akahira, M. (978). Thir orer asymptotic eciecy of maximum likelihoo estimator for multiparameter expoetial case. Rep. Uiv. Electro- Comm., 28, [43] Takeuchi, K. a Morimue, K. (985). Thir-orer eciecy of the extee maximum likelihoo estimators i a simultaeous equatio system. Ecoometrica, 53, [44] Taiguchi, M. (983). O the seco orer asymptotic eciecy of estimators of {8{

19 Gaussia ARMA Processes. A. Statist.,, [45] Taiguchi, M. (986). Thir orer asymptotic properties of maximum likelihoo estimators for Gaussia ARMA processes. J. Multivariate Aal., 8, -3. [46] Taiguchi, M. (987). Thir orer asymptotic properties of BLUE a LSE for a regressio moel with ARMA resiual. J. Time Ser. Aal., 8, -4. [47] Taiguchi, M. (99). Higher Orer Asymptotic Theory for Time Series Aalysis, Lecture Notes i Statistics, Vol.68. Spriger-Verlag, Berli. [48] Yoshia, N. (992). Asymptotic expasios of maximum likelihoo estimators for small iusios via the theory of Malliavi{Wataabe. Probab. Theory Relat. Fiels, 92, [49] Yoshia, N. (993). Asymptotic expasio of Bayes estimators for small iusios. Probab. Theory Relat. Fiels, 95, {9{