Journa o Mutivariate Anaysis 67, 122 (1998) Artice No. MV981757 Spherica Deconvoution* Dennis M. Heay, Jr. Dartmouth Coege Harrie Hendriks University o Nijmegen, Nijmegen, The Netherands and Peter T. Kim University o Gueph, Gueph, Ontario, Canada Received December 13, 1995; revised September 24, 1997 This paper proposes nonparametric deconvoution density estimation over S 2. Here we woud think o the S 2 eements o interest being corrupted by random SO(3) eements (rotations). The resuting density on the observations woud be a convoution o the SO(3) density with the true S 2 density. Conseuenty, the methodoogy, as in the Eucidean case, woud be to use Fourier anaysis on SO(3) and S 2, invoving rotationa and spherica harmonics, respectivey. We especiay consider the case where the deconvoution operator is a bounded operator owering the Soboev order by a inite amount. Consistency resuts are obtained with rates o convergence cacuated under the expected L 2 and Soboev suare norms that are proportionay inverse to some power o the sampe size. As an exampe we introduce the rotationa version o the Lapace distribution. 1998 Academic Press AMS subject cassiications primary 62G05; secondary 58G25. Key words and phrases consistency; density estimation; deconvoution; rotationa harmonics; rotationa Lapace distribution; Soboev spaces; spherica harmonics. 1. INTRODUCTION Directiona statistics invoves data anaysis on S 2, the two dimensiona unit sphere in Eucidean three space. There are now a arge number o statistica methodoogies avaiabe or directiona statistics; see, or exampe, * This research was supported in part by ARPA as administered by the AFOSR under Contracts AFOSR-90-0292 and DOD F4960-93-1-0567; by NSERC Canada, OGP46204; and by NATO, CRG951309. 1 0047-259X98 25.00 Copyright 1998 by Academic Press A rights o reproduction in any orm reserved.
2 HEALY, HENDRIKS, AND KIM the genera surveys ound in Fisher, Lewis, and Embeton (1987) and Jupp and Mardia (1989). The methodoogies deveoped or the spherica setting invove generaizing the statistica techniues or the Eucidean space. In most cases, this generaization appears uite natura, particuary when the probem is parametric. The probem however, is somewhat more invoved in the nonparametric situation and this incudes kerne density estimation or directiona data (see Beran, 1979; Ha, Watson, and Cabrera, 1987; Bai, Rao, and Zhao, 1988; Hendriks, 1990) and spine methods or directiona data (see Wahba, 1981). To date, as ar as we can te, there have been no attempts at nonparametric deconvoution density estimation on S 2, athough nonparametric deconvoution methodoogies or the Eucidean space abound; see, or exampe, Carro and Ha (1988), Fan (1991), and Digge and Ha (1993), as we as the reerences contained therein. Conseuenty, the ocus o this paper is to provide a spherica deconvoution techniue. The idea in deconvoution is to statisticay recover the density when observations consist o the true measurement corrupted by noise. In the S 2 case, we woud think o corruption by noise as a random rotation induced by the transitive group action o SO(3), the set o 3_3 rea orthogona matrices o determinant one. This o course is the direct generaization o the Eucidean version where noise is introduced in terms o the transitive group action o the additive group o transations. The genera techniue paraes existing Eucidean Fourier based methods. Interestingy enough, when such a strategy is adhered to, the mathematics is remarkaby simiar to that o the Eucidean version, with the exception that one must appropriatey modiy the Fourier anaysis. Indeed, the distributiona impact o incuding noise in the prescribed way resuts in a density that turns out to be the convoution o two unctions; one density is on SO(3) representing the noise, whie the other density is on S 2 representing the true measurement. I we then take the Fourier transorms, appropriatey deined, the convoution becomes ordinary matrix vector mutipication. By assuming that observations come rom this convoved density, we can empiricay estimate the Fourier transorm rom the avaiabe data. Foowing this, we woud need to invert and smooth the empirica transorm. It turns out that under reativey mid conditions, and conditions simiar to those o the Eucidean version, consistent nonparametric density estimators o the true measurement can be obtained. We now provide an overview o the paper. In Section 2, we provide the necessary Fourier anaysis toos that wi enabe us to adopt a technoogy transer rom Eucidean space to the spherica and rotationa spaces. This amounts to understanding Fourier anaysis on SO(3), as we as S 2. Indeed, the L 2 (S 2 ) Fourier basis, the spherica harmonics, is inherited rom the L 2 (SO(3)) Fourier basis, the
SPHERICAL DECONVOLUTION 3 rotationa harmonics, through the identiication o S 2 with the uotient space SO(3)SO(2), where SO(2), is the space o 2_2 rea orthogona matrices o determinant one. Conseuenty, convoution can be interpreted in terms o the distribution o a random vector on S 2 premutipied by a random SO(3) matrix. This transates to ordinary matrix vector mutipication in the Fourier domain so that by inverting the matrix and inverting the Fourier transorm, a genera expression or the convouted unction can be obtained. Section 3 statisticay impements this procedure assuming that the distribution o the random rotation is known and inversion makes sense. Simiar to the Eucidean setting, damping actors can be introduced to contro the accumuation o the higher order reuencies and this wi enabe us to obtain consistency resuts or the estimator. We show that L 2 rates o convergence can be obtained when smoothness in the underying density is assumed. We aso make generaizations to Soboev spaces. In such a setting, one can address the deconvoution probem in terms o Soboev order as we as evauate rates o convergence in terms o Soboev norms. Indeed deconvoution eads to convergence in expected suare Soboev norm, depending on the error distribution, o order proportionay inverse to some power o the sampe size, approaching the theoreticay optima order proportionay inverse to the sampe size. Some remarks with respect to the random SO(3) matrix are made. Indeed, a rotationa version o the Lapace distribution which appies to our method is introduced. We notice that convergence in appropriate Soboev norms impies consistency o density estimators together with their derivatives up to a given order. We point out that Soboev techniues have been used in directiona statistics in testing (see Gine, 1975; Jupp and Spurr, 1983, 1985). Section 4 contains the proos to the statements o Section 3. Prior to embarking upon the task at hand we point out that in principe, the anaysis o this paper can be extended to any p&1 dimensiona unit sphere, or p3. Indeed, one woud proceed in exacty the same way except that one woud have SO( p), the space o p_p rea orthogona matrices o determinant one, acting on S p&1. We do however point out that the compexity or the higher dimensiona spheres can be chaenging and is one o the reasons why we are restricting our anaysis to S 2. Nevertheess, it is a act that most noncommutative physica appications occur in this dimension. 2. FOURIER ANALYSIS ON SO(3) AND S 2 We wi provide a brie overview o Fourier anaysis on SO(3) and S 2. Most o the materia in expanded orm can be ound in Taman (1968).
4 HEALY, HENDRIKS, AND KIM Papers by Lo and Esheman (1979) and Wahba (1981) directy dea with simiar issues. Let cos, &sin, 0 cos % 0 sin % u(,)=\sin, cos, 0+, 0 1 0 a(%)=\ +, 0 0 1 &sin % 0 cos% where, #[0,2?), % #[0,?). The we known Euer ange decomposition says that any g # SO(3) can amost surey be uniuey written as g=u(,) a(%) u(), where, #[0,2?), % #[0,?), #[0,2?) and are otherwise known as the Euer anges. Consider the unction, where D 1 2 (u(,) a(%) u(.))=e &i 1, d 1 2 (cos %) e &i 2, (2.1) d 1 2 (cos %)=i 1 & 2 sin 2 & 1 %(1+cos %) 12_ 1 (& 12 2)! 2 [(+ 1 )! (& 1 )!] (+ 2 )!& _ d + 2 (cos %&1) + 1 (cos %+1) & 1, d(cos %) + 2 & 1, 2 and =0, 1,... The unctions D 1 2,& 1, 2, =0,1,..., are the eigenunctions o the Lapace Betrami operator on SO(3). Furthermore, [- 2+1 D 1 2 & 1, 2, =0,1,...] is a compete orthonorma basis or L 2 (SO(3)) with respect to the probabiity Haar measure and is sometimes reerred to as the rotationa harmonics (see Lo and Esheman, 1979). In addition, i we deine a (2+1)_(2+1) matrix by D (g)=[d 1 2 (g)], where & 1, 2, 0, and g # SO(3), these constitute the coection o ineuivaent irreducibe representations o SO(3) (see Taman, 1968).
SPHERICAL DECONVOLUTION 5 Let # L 2 (SO(3)). We deine the rotationa Fourier transorm on SO(3) by 1 2 = SO(3) (g) D 1 2 (g) dg, (2.2) where again we think o (2.2) as the matrix entries o the (2+1)_(2+1) matrix =[ 1 2 ], & 1, 2 and =0, 1,... The rotationa inversion can be obtained by (g)= 0 1, 2 =& 0 1, 2 =& = (2+1) 1 2 D 1 2 (g) (2+1) 1 2 D 2 1 (g &1 ), (2.3) or g # SO(3), where the overbar denotes compex conjugation. Stricty speaking, (2.3) shoud be interpreted in the L 2 sense athough with additiona smoothness conditions, it can hod pointwise. Spherica Fourier anaysis aso has simiar resuts. Any point on S 2 can be represented by =(cos, sin %, sin, sin %, cos %) t, where % #[0,?),, #[0,2?), and } t denotes transpose. Let (2+1)(&)! Y ( )=Y (,, %)=(&1) P 4?(+)! (cos %) e i,, (2.4) where % #[0,?),, #[0,2?), &, =0, 1,..., and P ( } ) are the Legendre unctions. The atter are deined as oows the Legendre poynomia is deined by P (x)= 1 d 2! dx (x2 &1), or 0 and x # [&1, 1]. Deine Legendre unctions by P 0 (x)=p (x) and P (x)=(1&x2 ) d 2 dx P (x), 0 where 0, 0 and x # [&1, 1]. For &0, deine them through the euation (&)! P &(x)=(&1) (+)! P (x),
6 HEALY, HENDRIKS, AND KIM or x # [&1, 1]. The eect o this choice is that Y =(&1) Y, & where 0, 0. The Legendre unctions satisy the recurrence reation (&+1) P +1 (x)&(2+1) xp (x)+(+) P&1 (x)=0. We note that we can think o (2.4) as the vector entries to the (2+1) vector Y ( )=[Y ( )], 0. In this situation [Y &, =0,1,...] orms a compete orthonorma basis over L 2 (S 2 ) and is sometimes reerred to as the spherica harmonics (see Taman, 1968). Let # L 2 (S 2 ). We deine the spherica Fourier transorm on S 2 by = ( ) Y ( ) d. (2.5) S 2 Again we think o (2.5) as the vector entries o the (2+1) vector =[ ], &, =0, 1,... The spherica inversion can be obtained by ( )= 0 =& Y ( ), (2.6) or # S 2. Again, stricty speaking, (2.6) shoud be interpreted in the L 2 sense athough with additiona smoothness conditions, it can hod pointwise. The mathematica reationship between SO(3) and S 2 is a beautiu resut in cassica anaysis. Topoogicay, we can identiy S 2 as the uotient set SO(3)SO(2). In terms o the Fourier basis, the reation can be described in terms o the Euer anges, where Y (%,,)= (2+1) D 0 (u(,) a(%) u()), (2.7) 4?, #[0,2?), % #[0,?), &, and=0, 1,... We note that athough an extra ange appears in the right hand side o (2.7), it is in act independent o. This oows rom going back to (2.1) and observing that when 2 =0, the expression becomes independent o. One o the most useu toos o Fourier anaysis is the act that convoution o two unctions in Fourier space turns out to be ordinary matrix
SPHERICAL DECONVOLUTION 7 mutipication. Indeed, et # L 2 (SO(3)) and h # L 2 (S 2 ). Deine the convoution V h( )= SO(3) (u) h(u &1 ) du, (2.8) or # S 2. We have the oowing convoution property. Lemma 2.1. Suppose # L 2 (SO(3)) and h # L 2 (S 2 ). Then In particuar, or each =0, 1,..., or a &. @ V h= h. (2.9) @ V h = j=& j h j, Proo. Let # L 2 (SO(3)). We note that ( @ V h) = V h( ) Y # S 2 ( ) d, or & and =0, 1,... Using the deinition o convoution, this is Note that # S 2 g # SO(3) (g) h(g &1 ) Y = SO(3) (g) S 2 h(g &1 ) Y = SO(3) (g) S 2 h( ) Y Y (g )= j ( ) dg d ( ) d dg (g ) d dg. Y j ( ) D j (g&1 ), (2.10) or # S 2 and g # SO(3). This can be deduced in the oowing way. As stated in the Introduction, we can regard S 2 as the uotient space SO(3)SO(2). Conseuenty, identiy # S 2 with a corresponding coset representative g # SO(3). Aong with the act that D is a group
8 HEALY, HENDRIKS, AND KIM homomorphism, i.e., D (gh)=d (g) D (h) or g, h # SO(3) and =0, 1,..., and that D (g &1 )=D (g)*, where } * represents conjugate transpose, we have Y (g )=[(2+1)4?]12 D (gg 0 ) =[(2+1)4?] 12 D 0 (g&1 g&1 ) =[(2+1)4?] 12 s=& =[(2+1)4?] 12 = s=& s=& Y s ( ) D s (g&1 ). D 0s (g&1 ) D s (g&1 ) D s0 (g ) D s (g&1 ) Substituting (2.10) into the above expression, we get (g) SO(3) h( ) j Y ( ) j D j (g&1 ) d dg S 2 = j SO(3) (g) D j (g&1 ) dg h j. Now note that D j(g &1 )=D j(g) or a g # SO(3). Thereore, ( @ V h) = (g) D j (g&1 ) dg h j j SO(3) = j SO(3) (g) D j (g) dg h j = j j h j, or a &, =0, 1,..., as reuired. K 3. DECONVOLUTION DENSITY ESTIMATION We can now describe the deconvoution probem. Consider Z==X, (3.1) where = is an SO(3) random eement and Z, X are S 2 random eements, with = and X assumed independent. The action is with respect to the
SPHERICAL DECONVOLUTION 9 transitive group action SO(3)_S 2 S 2, which consists o ordinary matrix mutipication. Let Z, =, X denote the densities o Z, =, X, respectivey. Through (3.1), the reation among the densities can be described by convoution, Z = = V X, as seen by oowing the amiiar corresponding Eucidean resut. Now consider X and Z as vectors given by ( X, ) and ( Z, ), respectivey, and as the matrix ( ) = =, j j. By (2.9) we can write X =[ = ]&1 Z, provided o course that the matrices [ = ]&1 exist or a =0, 1,... in a range o interest. In particuar, i X is bandimited with bandimit B, meaning that X vanishes or B, then we need ony consider beow the bandimit. Statisticay, (3.1) describes the non-eucidean anaogue o observations Z made up o the true measurement X, corrupted by noise =. Our interest is in the unknown X. It is assumed that = is known and that [ = ]&1 exists or a range o 's that concerns us. Since X is unknown, Z is aso unknown, hence Z is unknown. Nevertheess, we assume that a random sampe Z 1,..., Z n is avaiabe. This wi aow us to construct an empirica version n Z. By (2.9), a ogica estimator or X is thereore n, X =[ = ]&1 n, Z, (3.2) or =0, 1,... We can then produce a nonparametric deconvoution density estimator o X by (2.6), the spherica inversion. 3.1. Consistent Estimation Deine the empirica Fourier transorm on S 2 by n, Z, =1 n n j=1 Y (Z j), (3.3) which is an unbiased estimator o Z, or & and =0, 1,... Then by (3.2), n, X, =1 n n j=1 s=& = &1, s Y s (Z j), where &, =0, 1,..., and or ease o notation, we write &1=[ = = ]&1. We note that the empirica transorm as deined is the direct
10 HEALY, HENDRIKS, AND KIM nonabeian generaization o the empirica characteristic unction as deined on the rea ine; see, or exampe, Feuerverger and Mureika (1977). Choosing m=m(n) as n eads to the oowing nonparametric deconvoution density estimator o X on S 2, n X ( )= m =0 n =&{ 1 n j=1 s=& Y (Z = &1, s s j) = Y ( ), (3.4) where # S 2. For statistica motivation, we can rewrite (3.4) in another way. Deine K = n (, &)= m =0, s=& Y ( ) = &1, s Y s (&), where &, # S 2. Then an aternative way o writing (3.4) is n X ( )=1 n n j=1 K = n (, Z j), (3.5) where # S 2. Note that this resembes an ordinary kerne estimator in Eucidean space. For two seuences [a n ] and [c n ], symboize a n =O(c n )asnby a n <<c n,asn. I both a n <<c n and c n <<a n, we wi use the symbo a n B c n.for# L 2 (S 2 ) denote the L 2 -norm by & & 2 =[ S 2 ( ) 2 d ] 12 and or some operator A, et &Ax& &A& op =sup x{0 &x&. We have the oowing resuts. Theorem 3.1. Suppose & &1& = op<< u or some u>0. I X is the pointwise imit o its Fourier series and m 2u+4 =o(n) as n (m(n) ), then E n X ( )& X( ) 2 0, as n or a # S 2. In addition, i = is continuous, then convergence can be achieved or m 2u+2 =o(n). Theorem 3.2. Suppose & &1& = op<< u or some u>0. I X is s times dierentiabe with suare integrabe derivatives or some s1, then or m 2s+2u+2 B n as n. E & n X & X& 2 2 <<n&s(s+u+1),
SPHERICAL DECONVOLUTION 11 We remark that the above resuts are stated in such a way that they resembe as cosey as possiby kerne estimators in the Eucidean case. In particuar, the roe o m is that o the reciproca o the bandwidth parameter or ordinary kerne estimators. Conseuenty, in addition to consistency, rate optimization or m is aso exhibited. 3.2. Generaization to Soboev Functions In a technica sense, the most natura setting or using Fourier techniues is Soboev spaces. In particuar, this space o unctions extends the L 2 space with a norm depending on smoothness properties caed the Soboev norm. Estimation in Soboev spaces impies estimation o certain partia derivatives (see resuts ii and vi beow). Some recognition in the statistica iterature as to its useuness has been addressed or Eucidean deconvoution density estimation (see, or exampe, Eromovich, 1997). Conseuenty, we woud ike to present the resuts o the previous section in this context. On the space C (S 2 ) o C unctions on S 2 one may consider the so-caed Soboev norm &}& Hs o order s deined on a unction g( )=g^ Y ( ) by &g& 2 H s = (1+(+1)) s g^ 2., Reca that Y, =&,...,, are eigenunctions o the LapaceBetrami operator 2 with eigenvaues (+1). Conseuenty one has the property that &(1+2) g& Hs =&g& Hs+2.LetH s (S 2 ) denote the competion o C (S 2 ) with respect to the Soboev H s norm o order s. Thus each eement o H s (S 2 ) has a we determined Fourier transorm. It is cear that (1+2) H s+2 (S 2 ) H s (S 2 ) is an isometry. The oowing resuts are we known. (i) As normed spaces there is the euaity H 0 (S 2 )=L 2 (S 2 ). (ii) Soboev emma. For s>1, or each eement o H s (S 2 ), the Fourier transorm is absoutey summabe and thereore determines a continuous unction and the corresponding map H s (S 2 ) C 0 (S 2 ) is continuous with respect to the sup-norm in C 0 (S 2 ). More generay, et k0 be an integer; then there is a uniue map H s+k (S 2 ) C k (S 2 ) respecting the Fourier transorm. This map is continuous with respect to the C k -topoogy on C k (S 2 ), which eads to the act that density estimation in H s+k (S 2 ) impies the estimation o a derivatives up to order k in the sup-norm sense. (iii) For any s there is a duaity H &s (S 2 )_H s (S 2 ) R deined by ( P Y, Y )= P. In particuar (P, ) &P& H&s & & Hs. This duaity deines an isometry between H &s (S 2 ) and the dua o H s (S 2 ).
12 HEALY, HENDRIKS, AND KIM (iv) Any (Bore) probabiity distribution P on S 2 has a we determined Fourier transorm with P =E(Y ). According to the Soboev emma P Y # H &s(s 2 ) or s>1. Moreover, et g= g^ Y # H s(s 2 ). Then g represents a continuous unction, and we have g( ) P(d )= g^ P =(P, g). (v) Let s1 be an integer and g be a rea vaued unction on S 2 which is s times dierentiabe with suare integrabe sth order partia derivatives. Then the Fourier transorm o g is the Fourier transorm o an eement o H s (S 2 ). (vi) Let D be a partia dierentia operator o order d with smooth coeicients. Then D H s (S 2 ) H s&d (S 2 ) is a continuous operator. We remark that together with resut i., this means that density estimation in H k (S 2 ) is euivaent to the estimation o a partia derivatives up to order k in L 2 -sense. Notice that the conditions in Theorems 3.1 and 3.2 impy that the inverse = &1 o the convoution operator g [ = V g has the property that or some C>0, 2 ( = &1 g) 2 = 1 } = &1, 1 2 g^ 2 2} C 2 (1+(+1)) u g^ 2 2, 2 so that = &1 H s (S 2 ) H s&u (S 2 ) is a continuous operator (with operator norm ess than C). We have the oowing generaization o Theorem 3.2. Theorem 3.3. Suppose that or some u>0, = &1 H t (S 2 ) H t&u (S 2 ) is a continuous operator, or euivaenty & &1& = op<< u. Suppose X # H s (S 2 ). Let v<s and =2s+2u+2. Then or m B n 1 as n. E & n X & X& 2 H v <<n &(s&v)(s+u+1), Notice that this theorem impies a convergence theorem or derivatives o X. In particuar or a dierentia operator D o order d with smooth coeicients, we wi have E &D n X &D X& 2 H v&d <<n &(s&v)(s+u+1), which gives or v=d convergence in the L 2 sense and or v&d>1 convergence in the sup-norm sense. The appication in the spirit o Theorem 3.1 woud be or the parameter v=1+$ or any $>0, and s>v so that in particuar X is a continuous unction, and its Fourier transorm is uniormy absoutey summabe.
SPHERICAL DECONVOLUTION 13 Coroary 3.4. Suppose X # H s (S 2 ) or some s>1, and et 1<v<s. Let =2s+2u+2; then or m B n 1 we have E sup n X ( )& X( ) 2 <<E & n X & X& 2 H v <<n &(s&v)(s+u+1). In comparison with Theorem 3.1, here the smoothness condition on X is stronger, which in turn eads to a known rate o convergence. 3.3. Remarks on the Error and the Rotationa Lapace Distribution The uestion that naturay arises concerns the type o errors = that woud ead to the existence o = &1 or =0, 1, 2,... At this point we oer two extremes. At one extreme is the uniorm distribution on SO(3). In this case deconvoution is not possibe since = =0 or a >0. The other extreme woud be point mass at the unit eement denoted by $ e. Then =I = 2+1, or =0, 1,..., where I denotes the identity matrix reative to the dimension. In such a case, (3.4) is exacty the density estimator proposed by Hendriks (1990) or S 2. This is not surprising since i = is point mass at the unit eement o SO(3) then the observed data is the true measurement. Conseuenty, an interesting error distribution shoud be between these two extremes with a ree parameter to measure concentration. We present a new amiy o probabiity distributions on SO(3) with the property that the convoution o a probabiity distribution on S 2 with a member o this amiy raises the Soboev order by exacty 2. Athough the order o smoothness 2 coud be repaced by any other positive order, in this case we are abe to give an expicit anaysis o the amiy. The amiy has an exact anaogy with the amiy o so-caed Lapace or douby exponentia distributions on the rea ine, given by the density unction 1 2_ exp(& x _), whose Fourier transorm (characteristic unction) is given by (1+_ 2 t 2 ) &1, where _>0. What we are aiming or is an error distribution = on SO(3) with the property that or g # H s (S 2 ) = V g( )= = V g^ Y ( )= 1 1+_ 2 (+1) g^ Y ( ),
14 HEALY, HENDRIKS, AND KIM where _>0. As this behavior is anaogous to the behavior o the amiy o Lapace distributions with respect to convoution, we propose to name this amiy o distributions = the rotationa Lapace distributions. Thus = V g # H s+2 (S 2 ) and the convoution with = gives a map H s (S 2 ) H s+2 (S 2 ) with a continuous inverse, namey the partia dierentia operator (1+_ 2 2). From the convoution property (2.9), it oows that =, j =(1+_2 (+1)) &1 $ j, so that by the inverse rotationa Fourier transorm = = 0 =& (1+_ 2 (+1)) &1 (2+1) D. (3.6) We have the oowing, where or a given g # SO(3), a 3_3 matrix, the trace o g is denoted by tr(g). Theorem 3.5. The density unction = with respect to the uniorm probabiity measure o the rotationa Lapace distribution with parameter _ is given by = (g)= _&2? cos(?) cos((?&r)), sin(r2) where =- (14)&_ &2 and r #[0,?] is the rotation ange o g given by the reation tr( g)=1+2 cos(r). For _<2, is purey imaginary, and one may use the reation that cos(ix)=cosh(x), where cosh denotes the hyperboic cosine unction. Some background materia as we as the proo o Theorem 3.5 is presented in Section 4.1. As a reeree thoughtuy points out, because the tangent space at the unit eement o SO(3) is the space o 3_3 skew symmetric matrices, which we wi denote by so(3), an aternative parameterization o SO(3) is governed by the exponentia map exp so(3) SO(3) and can be represented by exp(a)=cos(&a&) I+ sin(&a&) &a& A+ 1&cos(&a&) &a& 2 aa t, where A # so(3) and a=(&a 23, A 13,&A 12 ) t. This means that or the rotation exp(a)#so(3), the rotation ange is &a& about the axis a&a&. Conseuenty, with respect to Theorem 3.5, r(exp(a))=&a& so that the atter is simpy the rotation ange.
SPHERICAL DECONVOLUTION 15 4. PROOFS In this section we wi prove the statements given in Section 3. We irst compute the asymptotic variance. Lemma 4.1. Suppose or some C>0, the operator norms satisy & = &1& opc u or a =0,1,...Then 2u+4 sup Var( n ( ))<<m X # S 2 n and Var( n X ( )) d <<m2u+2. S 2 n as n. I in addition = is continuous, then 2u+2 sup Var( n ( ))<<m X. # S 2 n Proo. We note that Var( n ( ))=1 X n Var(K = (, Z))1 n n EK = (, Z) K = n n (, Z), or # S 2, where Z denotes the random S 2 eement =X. Now, EK = (, Z) K = n n (, Z)sup, & sup, & } K = (, &) K = n n (, &) m =0 m sup =0 m sup =0 Y ( ) s m =(m+1) 2 m Y ( ) 2 m (2+1) =0 =0 2 Y (&) = &1, s s } =0 } s C 2 2u Y s (&) 2 C 2 2u (2+)<<m 2u+4. s 2 Y (&) = &1, s s } On the other hand, i the density = is continuous, then Z is continuous, thereore bounded, so that S 2 m K = (, z) K = (, z) n n Z(z) dz<< =0 <<m 2u+2, s=&} =& 2 Y ( ) = &1, s}
16 HEALY, HENDRIKS, AND KIM where the bound is independent o # S 2,asn. As a conseuence S 2 Var( n X ( )) d <<m2u+2. K n Proo o Theorem 3.1. Under the conditions o the theorem, note that E n ( )=EK = X n (, Z) m = =0 m = =0, s=& =& Y ( ) = &1, s Z, s Y ( ) X,, (4.1) or # S 2. According to the hypothesis, this converges to X ( ) asn. Conseuenty, because the pointwise mean suared error decomposes in terms o the variance and the suare o the bias, consistency oows in ight o the above and Lemma 4.1. K With additiona smoothness conditions, more expicit bounds on the bias can be obtained. We have the oowing. Lemma 4.2. Suppose X is s-times dierentiabe with suare integrabe derivatives, where s1. Then as n. & X &E n X &2 2 <<m&2s, Proo. By an argument simiar to that o Hendriks (1990, p. 842) or M=S 2, aong with (4.1) pus the act that the eigenvaues corresponding to the LapaceBetrami operator on S 2 are (+1) or Y, &, 0, we have & X &E n X &2 2 & (s) X &2 2 m 2s, as m, where (s) X reers to some sth derivative o X, or s1. K Proo o Theorem 3.2. For the genera case, by appying Fubini and putting Lemmas 4.2 and 4.1 together, we get
SPHERICAL DECONVOLUTION 17 E & n & X X& 2 = 2 [E n ( )& X X( ) 2 ]d S 2 = S 2 [Var( n X ( ))+ X( )&E n X ( ) 2 ]d m2u+2 +& X &E n X n &2 2 << m2u+2 + 1 n m 2s, as n or s1. This rate is optimized when Conseuenty, as n. K m B n 1(2s+2u+2). E & n X & X& 2 2 <<n&s(s+u+1), In order to prove the resuts o Section 3.2 we need a generaization o Lemma 4.2. Lemma 4.3. as n. Proo. Suppose X # H s (S 2 ), and et v<s. Then Notice that & X &E n X &2 H v = >m & X &E n X &2 H v <<m &2(s&v), (1+(+1)) v X 2 (1+m(m+1)) &(s&v) >m (1+(+1)) s X 2 (1+m(m+1)) &(s&v) & & 2 H s <<m &2(s&v). Now we must investigate the behavior o & n X &E n X &2 H v. It is described in the oowing emma, which is anaogous to Lemma 4.1. K Lemma 4.4. Then Suppose = &1 H t (S 2 ) H t&u (S 2 ) is a continuous operator. E & n &E n X X &2 H v << m2(v+u+1), n as n.
18 HEALY, HENDRIKS, AND KIM Proo. As n X = = &1 n Z, we have the ineuaity & n X &E n X &2 H v << & n Z &E n Z &2 H v+u. But E & n Z&E n Z& 2 H v+u = 1 n sup m = 1 n Var[(1+(+1)) (v+u)2 Y (Z)] 1 n m m [(1+(+1)) (v+u)2 Y ( )]2 (1+(+1)) (v+u) (2+1)<< m2v+2u+2. K n Proo o Theorem 3.3. decomposition As in the proo o Theorem 3.2 we have the E & n X & X& 2 H v =E & n X &E n X &2 H v +&E n X & X& 2 H v as n. This rate is optimized when Conseuenty << m2v+2u+2 +m &2(s&v) n m=n 1(2s+2u+2). E & n X& X & 2 H v <<n &(s&v)(s+u+1), as n. K 4.1. Rotationa Lapace Distributions In this subsection we present the background and the proo o Theorem 3.5. In order to investigate the expression (3.6) we need some preparation. First, we make a deinite choice o the LapaceBetrami operator on SO(3), which is we deined up to a positive scaar actor, by the condition that the eigenunctions D j be eigenunctions or the eigenvaue (+1). Second, we make a choice o radia coordinate with respect to the unit eement o SO(3) as the unction r SO(3) [0,?] given by the condition that tr(g)=2 cos(r(g))+1 so that the radia coordinate o a rotation about some axis is the ange (in radians) o rotation chosen between 0 and?, independenty o the direction o the axis. A conjugate invariant unction on SO(3) wi be a unction in
SPHERICAL DECONVOLUTION 19 the radia coordinate and the LapaceBetrami operator appied to such a unction wi be given by &2(r)= \ d 2 d dr 2+cos(r2) sin(r2) dr+ (r). (4.2) The unction g [ tr(g)=2 cos(r)+1 is an eigenunction o 2 or the eigenvaue 2. Since tr(g)= D 1 (g) this shows that the above standardization o 2 is correct. The integration o unctions o the radia coordinate over SO(3) with respect to the uniorm (invariant) probabiity density, is SO(3) = 2?? 0 (r) sin(r2) 2 dr. Proo o Theorem 3.5. Notice that the Fourier transorm o the point measure $ e at the unit eement e o SO(3) is given by ($ e) j=d j(e)=$ j. The inverse rotationa Fourier transorm then eads to $ e (g)= 0 =& (2+1) D (g &1 ) (vaid in H s (SO(3)) or s<&32), rom which one obtains the oowing partia dierentia euation or = (1+_ 2 2) = =$ e. We may assume that the soution to this euation is conjugate invariant, and thus is a unction o the radia coordinate r. The soution is smooth or r{0 and shoud be invariant under the substitution o r with 2?&r or r cose to?. Since the dierentia operator (4.2) is invariant under the substitution o r with 2?&r, its soution space spits in a 1-dimensiona space o unctions even in?&r and a 1-dimensiona space o unctions odd in?&r. Thus the unction we ook or wi be a soution which is even in?&r, satisying, or 0<r<2?, the dierentia euation d 2 dr =+ cos(r2) d 2 sin(r2) dr =&_ &2 = =0. It is a simpe matter to veriy that such a soution is the unction F given by F(r)= cos((?&r)), sin(r2)
20 HEALY, HENDRIKS, AND KIM where =- 14&_ &2. Moreover it is an eementary matter to veriy that SO(3) F= 2?? 0 F(r) sin(r2) 2 dr= cos(?) _ &2?. Now = is determined by the condition that it have integra 1, which eads to the unction given in the theorem. Notice that i is rea, it beongs to the interva [0, 12], so that = (r) then is positive or r #[0,2?]. I is not rea, then it is purey imaginary, and cos((?&r))=cosh(im()(?&r))>0. And then aso, = (r) is positive or r #[0,2?]. K We remark that the dierentia euation or unctions R 3 R, deined on Eucidean space, (1+_ 2 2) =$ 0, has as a soution a unction with Fourier transorm ( y)=(1+_ 2 &y& 2 ) &1 and that this is the characteristic unction o the probabiity density given by (x)= 1 4?_ 2 &x& exp(&&x&_). Up to a actor _ 2 this unction is known in physics as the Eucidean propagator o an interaction intermediated by a partice o mass _ &1. Notice that the behavior o the above density at the singuar point 0 as a unction o the ``radia'' coordinate &x& is exacty ike the behavior o the density o the rotationa Lapace distribution in r with respect to the Lebesgue measure on SO(3). The extra actor 8? 2 is the voume o SO(3). Another interpretation o the error distribution oows rom the Mein transorm (1+_ 2 2) &u = 1 1(u) tu&1 exp(&t(1+_ 2 2)) dt = 1 1(u) tu&1 exp(&t) exp(&t_ 2 2) dt. It means that the operator (1+_ 2 2) &u is obtainabe rom Brownian motion with diusion coeicient _, during random Gamma-distributed time (with parameter u). For u=1 this is an exponentiay distributed time with expected vaue 1.
SPHERICAL DECONVOLUTION 21 The operator (1+_ 2 2) aso governs easticity theory (without the torue part). See Grenander (1976, p. 420) or urther stochastica use o the metaphor o eastic deormations. ACKNOWLEDGMENTS We thank the editor and a reeree or their comments that ed to an improvement o an earier version o this paper. In addition, parts o this research were competed whie the third author was visiting the Department o Appied Statistics, Yonsei University, Seou 120-749, Korea. He takes this opportunity to acknowedge Yonsei University and, in particuar, Proessor B. Sam Yoo or their hospitaity during the a o 1997. REFERENCES 1. Z. D. Bai, C. R. Rao, and L. C. Zhao,, Kerne estimators o density unction o directiona data, J. Mutivariate Ana. 27 (1988), 2439. 2. R. Beran, Exponentia modes or directiona data, Ann. Statist. 6 (1979), 11621178. 3. R. J. Carro, and P. Ha, Optima rates o convergence or deconvoving a density, J. Amer. Statist. Assoc. 83 (1988), 11841186. 4. T. Chang, Spherica regression, Ann. Statist. 14 (1986), 907924. 5. G. R. Chapman, G. Chen, and P. T. Kim, Assessing geometric integrity using spherica regression techniues, Statist. Sinica 5 (1995), 173220. 6. P. J. Digge and P. Ha, A Fourier approach to nonparametric deconvoution o a density, J. Roy. Statist. Soc. Ser. B 55 (1993), 523531. 7. S. Eromovich, Density estimation or the case o supersmooth measurement error, J. Amer. Statist. Assoc. 92 (1997), 526535. 8. J. Fan, Goba behavior o deconvoution kerne estimates, Statist. Sinica 1 (1991), 541551. 9. A. Feuerverger and R. A. Mureika, The empirica characteristic unction and its appications, Ann. Statist. 5 (1977), 8897. 10. N. I. Fisher, T. Lewis, and B. J. Embeton, ``Statistica Anaysis o Spherica Data,'' Cambridge Univ. Press, Cambridge 1987. 11. M. E. Gine, Invariant tests or uniormity on compact Riemannian maniods based on Soboev norms, Ann. Statist. 3 (1975), 12431266. 12. U. Grenander, ``Pattern Synthesis,'' Lecture in Pattern Theory, Vo. 1, Springer-Verag, Berin 1976. 13. P. Ha, G. Watson, and J. Cabrera, Kerne density estimation with spherica data, Biometrika 74 (1987), 751762. 14. H. Hendriks, Nonparametric estimation o a probabiity density on a Riemannian maniod using Fourier expansions, Ann. Statist. 18 (1990), 832849. 15. P. E. Jupp and K. V. Mardia, A uniied view o the theory o directiona statistics, 19751988, Internat. Statist. Rev. 57 (1989), 261294. 16. P. E. Jupp and B. D. Spurr, Soboev tests or symmetry o directiona data, Ann. Statist. 11 (1985), 12251231. 17. P. E. Jupp and B. D. Spurr, Soboev tests or independence o directions, Ann. Statist. 13 (1983), 11401155. 18. C. G. Khatri and K. V. Mardia, The von MisesFisher distribution in orientation statistics, J. Roy. Statist. Soc. Ser. B 39 (1977), 95106.
22 HEALY, HENDRIKS, AND KIM 19. J. T. Lo and L. R. Esheman, Exponentia Fourier densities on SO(3) and optima estimation and detection or rotationa processes, SIAM J. App. Math. 36 (1979), 7382. 20. L. P. Rivest, Spherica regression or concentrated Fishervon Mises distributions, Ann. Statist. 17 (1989), 307317. 21. J. D. Taman, ``Specia Functions A Group Theoretic Approach,'' Benjamin, New York 1968. 22. G. Wahba, Spine interpoation and smoothing on the sphere, SIAM J. Sci. Statist. Comput. 2 (1981), 516.