9512.net

甜梦文库

甜梦文库

当前位置：首页 >> >> # Group cohomology and the singularities of the Selberg zeta function associated to a Kleinia

Annals of Mathematics, 149 (1999), 627–689

Group cohomology and the singularities of the Selberg zeta function associated to a Kleinian group

By Ulrich Bunke and Martin Olbric

h

arXiv:dg-ga/9603003v2 1 Mar 1999

Contents 1. Introduction 1.1. The Selberg zeta function 1.2. Singularities and spectrum 1.3. Singularities and group cohomology 1.4. The main result 1.5. The extension map 2. Restriction, extension, and the scattering matrix 2.1. Basic notions 2.2. Holomorphic functions to topological vector spaces 2.3. The push down 2.4. Elementary properties of extλ 2.5. The scattering matrix 2.6. Extension of hyperfunctions and the embedding trick 3. Green’s formula and applications 3.1. Asymptotics of Poisson transforms 3.2. An orthogonality result 3.3. Miscellaneous results 4. Cohomology 4.1. Hyperfunctions with parameters 4.2. Acyclic resolutions 4.3. Computation of H ? (Γ, Oλ C ?ω (Λ)) 4.4. The Γ-modules O(λ,k) C ?ω (Λ) 5. The singularities of the Selberg zeta function 5.1. The embedding trick 5.2. Singularities and cohomology

628

ULRICH BUNKE AND MARTIN OLBRICH

1. Introduction Let G := SO(1, n)0 denote the group of orientation-preserving isometries of the n-dimensional hyperbolic space X := H n equipped with the Riemannian metric of constant sectional curvature ?1. We consider a convex cocompact, torsion-free discrete subgroup Γ ? G. The quotient Y := Γ\X is a complete hyperbolic manifold, and we assume that vol(Y ) = ∞. The Selberg zeta function ZS (s), s ∈ C, associated to this geometric situation encodes the length spectrum of closed geodesics of Y together with the eigenvalues of their Poincar? e maps. Note that ZS (s) is given by an Euler product on some half-plane Re(s) > c, and it has a meromorphic continuation to the complex plane. The goal of the present paper is a description of the singularities of the Selberg zeta function in terms of the group cohomology of Γ with coe?cients in certain in?nite dimensional representations. Such a relation was conjectured by Patterson [37]. 1.1. The Selberg zeta function. In order to ?x our conventions we de?ne ZS (s) in terms of group theory. Let g = k ⊕ p be a Cartan decomposition of the Lie algebra of G, where k is the Lie algebra of a maximal compact subgroup K ? G, K ? = SO(n). We ?x a one-dimensional subspace a ? p and let M ? K , M ? = SO(n ? 1), denote the centralizer of a. The Riemannian metric of X induces a metric on a. We ?x an isometry a ? = R. Let a+ denote the half-space corresponding to the ray [0, ∞) and set A := exp(a), A+ := exp(a+ ). By n ? g we denote the positive 1 tr ad(H )|n . The isometry root space of a in g. For H ∈ a we set ρ(H ) := 2 n?1 ? ? a = R identi?es ρ with 2 . By the Cartan decomposition G = KA+ K , any element g ∈ G can be written as g = hag k, h, k ∈ K , where ag ∈ A+ is uniquely determined. We have ag = edist(O,gO) , where O = K ∈ X is the origin of X = G/K , and dist denotes the hyperbolic distance. The basic quantity associated with Γ is its exponent δΓ which measures the growth of Γ at in?nity. ? R of Γ is de?ned to be the De?nition 1.1. The exponent δΓ ∈ a? = smallest number such that the Poincar? e series (1)

g ∈Γ (s+ρ) a? g

converges for all s > δΓ .

1 Although we do not use it in the present paper note that δΓ + n? 2 is equal to the Hausdor? dimension of the limit set of Γ (see [35], [47]).

SELBERG ZETA FUNCTION

629

Any element g ∈ Γ is conjugated in G to an element of the form m(g)a(g) ∈ M A+ , where a(g) is unique. By lg := log(a(g)) we denote the length of the closed geodesic of Y corresponding to the conjugacy class of g ∈ Γ. Let C Γ denote the set of conjugacy classes [g] = 1 of Γ. If [g] ∈ C Γ, then nΓ (g) ∈ N is the multiplicity of [g], i.e., the largest number k ∈ N such that [g] = [hk ] for some [h] ∈ C Γ. If A : V → V is a linear homomorphism of a complex vector space V , then by S k A : S k V → S k V we denote its kth symmetric power. De?nition 1.2. The Selberg zeta function ZS (s), s ∈ C, Re(s) > δΓ , associated to Γ is de?ned by the in?nite product

∞

(2)

ZS (s) :=

[g ]∈C Γ,nΓ (g )=1 k =0

1 det 1 ? e?(s+ρ)lg S k (Ad(m(g)a(g))? |n )

.

Remark. We de?ned the Selberg zeta function such that its critical line is {Re(s) = 0}. In the literature the convention is often such that the critical 1 line is at ρ = n? 2 . The same applies to our de?nition of δΓ which di?ers by ρ from the usual convention. In [38] (see also [42]) it was shown that the in?nite product converges for Re(s) > δΓ , and that the Selberg zeta function has a meromorphic continuation to all of C. In the special case of surfaces this was also proved in [17]. Partial results concerning the logarithmic derivative of the Selberg zeta function have been obtained in [34] for δΓ < 0 and in [41] in the general case. 1.2. Singularities and spectrum. The Selberg zeta function is a meromorphic function de?ned in terms of a classical Hamiltonian system, namely the geodesic ?ow on the unit sphere bundle SY of Y . Philosophically, the singularities of the Selberg zeta function should be considered as quantum numbers of an associated quantum mechanical system. One way to quantize the geodesic ?ow is to take as the Hamiltonian the Laplace-Beltrami operator ?Y acting on functions on Y . In order to explain this philosophy let Y = Γ\G/K for a moment be a compact locally symmetric space of rank one. Then the sphere bundle of Y can be written as SY = Γ\G/M . If (σ, Vσ ) is a ?nite-dimensional unitary representation of M , then we consider the bundle V (σ ) := Γ\G ×M Vσ over SY . The geodesic ?ow admits a lift to V (σ ) and gives rise to a more general Selberg zeta function ZS (s, σ ) which also encodes the holonomy in V (σ ) of the ?ow along the closed geodesics. The Selberg zeta function ZS (s) de?ned above corresponds to the trivial representation of M . It was shown in [13] that ZS (s, σ ) admits a meromorphic continuation to all of C. In this generality a description of the singularities of ZS (s, σ ) was ?rst obtained by [21] (see also [48] for a closely related Selberg zeta function).

630

ULRICH BUNKE AND MARTIN OLBRICH

Remark. The case of a Riemann surface is classical. The Selberg zeta functions for general rank-one symmetric spaces and trivial σ have been discussed in [14]. For a detailed account of the literature see [5]. The description of the singularities of ZS (s, σ ) given in [21] corresponds to a di?erent method of quantization of the geodesic ?ow (see subsection 1.3). The spectral description of the singularities of ZS (s, σ ) uses di?erential operators acting on sections of bundles on Y . One distinguishes between two types of singularities, so-called topological and spectral singularities. If σ is trivial then the spectral singularities are connected with the eigenvalues of the Laplace-Beltrami operator ?Y . The topological singularities depend on the spectrum of the Laplace-Beltrami operator on the compact dual symmetric space to X . For general σ we found the corresponding quantum mechanical system in [5]; i.e., we determined the locally homogeneous vector bundle on Y together with a corresponding locally invariant di?erential operator whose eigenvalues are responsible for the spectral singularities of ZS (s, σ ). The analytic continuation of this operator to the compact dual symmetric space gives rise to the topological singularities. We now return to our present case that Γ ? SO(1, n)0 is convex cocompact, Y is a noncompact hyperbolic manifold, and σ is the trivial representation of M . It was shown in [28] that the spectrum of ?Y consists of ?nitely many 1 2 n?1 2 isolated eigenvalues in the interval [( n? 2 ) ? |δΓ |δΓ , ( 2 ) ). Moreover, in [29] the same authors show that the remaining spectrum of ?Y is the absolute con1 2 tinuous spectrum of in?nite multiplicity in the interval [( n? 2 ) , ∞). It turns out that the eigenvalues of ?Y are responsible for singularities of ZS (s) as in the cocompact case. This stage of understanding is not satisfactory. On the one hand ZS (s) may have more singularities. On the other hand the continuous spectrum was neglected. A ?ner investigation of the continuous spectrum can be based on study of 1 2 the resolvent kernel, i.e. the distributional kernel of the inverse (?Y ? ( n? 2 ) + 2 ? 1 λ ) . It is initially de?ned for Re(λ) ? 0. A continuation of this kernel up to the imaginary axis implies absolute continuity of the essential spectrum by the limiting absorption principle (see [39] and for surfaces also [12], [10]). But this kernel behaves much better. It was shown in [32] that it has a meromorphic continuation to the whole complex plane (for surfaces see also [9], [1]). The poles of this continuation with positive real part correspond to the eigenvalues of ?Y . The poles with nonpositive real part are called resonances. Let us consider the resonances as sorts of eigenvalues associated to the continuous spectrum. Then they lead to singularities of ZS (s) in the same way as true eigenvalues. In detail, the spectral description of the singularities of ZS (s) was worked out in [38] for even dimensions n.

SELBERG ZETA FUNCTION

631

1.3. Singularities and group cohomology. An important feature of the spectral description of the singularities of the Selberg zeta function is the distinction between spectral and topological singularities. By now there are two approaches to describe all singularities of the Selberg zeta function in a uniform way, avoiding a separation of topological and spectral singularities. We will explain these approaches again for ZS (s, σ ) in the case where Y is a compact locally symmetric space of rank one. The ?rst approach was worked out in [21] (ideas can be traced back to [16], [36]) and corresponds to a quantization which is di?erent from the one considered for the spectral description. Here one considers the cohomology of the σ -twisted tangential de Rham complex (called tangential cohomology), i.e. the restriction of the de Rham complex of SY to the stable foliation twisted with V (σ ). This complex is equivariant with respect to the ?ow. The order of the singularity of ZS (s, σ ) at s = λ is related to the Euler characteristic of the λ + ρ-eigenspace of the ?ow generator on the tangential cohomology. The tangential cohomology comes with a natural topology, and it is still an open problem to show that this topology is Hausdor?. Therefore, in [21] the result is phrased in terms of representation theory, in particular in terms of Lie algebra cohomology of n with coe?cients in the Harish-Chandra modules of the unitary representations occurring in the decomposition of the right regular representation of G on L2 (Γ\G). The second approach was proposed by S. Patterson [37]. The parameters σ and λ ∈ C ?x a principal series representation (π σ,λ , H σ,λ ) of G. The space H σ,λ can be realized as the space of sections of a homogeneous vector bundle V (σλ ) := G ×P Vσλ , where P := M AN , N := exp(n), and σλ is the representation of P on Vσ given by P = M AN ? man → aρ?λ σ (m). Taking distribution sections we obtain the distribution globalization of σ,λ this principal series representation which we denote by H?∞ . If V is a com? plex representation of Γ, then H (Γ, V ) denotes group cohomology of Γ with coe?cients in V . The specialization of Patterson’s conjecture to cocompact Γ is:

σ,λ (i) dim H ? (Γ, H?∞ ) < ∞, σ,λ (ii) χ(Γ, H?∞ ) := ∞ i i=0 (?1) σ,λ dim H i (Γ, H?∞ ) = 0, and

σ,λ σ,λ i i (iii) ?χ1 (Γ, H?∞ ) := ? ∞ i=0 (?1) i dim H (Γ, H?∞ ) is the order of ZS (s, σ ) at s = λ (a pole has negative and a zero has positive order).

This conjecture was proved in [4] and [7] (with a slight modi?cation for s = 0) by clarifying the relation between both approaches. In (i)–(iii) one can σ,λ σ,λ replace H?∞ by the space H? ω of hyperfunction sections.

632 (C ? , d) e.g.

ULRICH BUNKE AND MARTIN OLBRICH

One way to de?ne group cohomology is to write down an explicit complex such that H ? (Γ, V ) is the cohomology of this complex. One can take

C p := {f : Γ × . . . Γ → V |f (gg0 , . . . , ggp ) = gf (g0 , . . . , gp )}

p+1 times

and (df )(g0 , . . . , gp+1 ) :=

p+1 i=0

(?1)i f (g0 , . . . , g ˇi , . . . , gp+1 ) .

Alternatively one can de?ne group cohomology as the right derived functor of the left exact functor from the category of complex representations of Γ to complex vector spaces which takes in each representation the subspace of Γ-invariant vectors. By homological algebra one can compute group cohomology using acyclic resolutions. To ?nd workable acyclic resolutions for the representations of interest is one of the main goals of the present paper. Let now Γ ? G be convex cocompact such that Y is a noncompact manifold. The space G/P can be identi?ed with the geodesic boundary ?X of X . There is a Γ-invariant partition ?X = ? ∪ Λ, where Λ is the limit set, and ? = ? is the domain of discontinuity for Γ with compact quotient B := Γ\?. According to the conjecture of Patterson for the convex cocompact case one σ,λ should replace H?∞ by the Γ-submodule of distribution sections of V (σλ ) with support on the limit set Λ. The main aim of this paper is to prove the conjecture of Patterson for convex cocompact Γ ? SO(1, n)0 and trivial σ up to two modi?cations which we will now describe. The ?rst modi?cation is that we consider hyperfunctions instead of distributions. From the technical point of view hyperfunctions are more natural and easier to handle. In fact, in several places we use the ?abbiness of the sheaf of hyperfunctions. Hyperfunctions also appear in a natural way as boundary values of eigenfunctions of ?X . In order to obtain distribution boundary values one would have to require growth conditions. Guided by the experience with cocompact groups and by the fact that the spaces of Γ-invariant hyperfunctions and distributions with support on Λ coincide, we believe that the cohomology groups are insensitive to replacing hyperfunctions by distributions (note that the situation is di?erent if Γ has parabolic elements; see [7]). The second modi?cation is in fact already necessary in the cocompact case to get things right at the point s = 0. Note that the principal series representations H σ,λ come as a holomorphic family parametrized by λ ∈ C. σ,λ In the conjecture we replace H? ω by the representation of Γ on the space of σ,? Taylor series of length k > 0 at λ of holomorphic families C ? ? → f? ∈ H? ω such that supp(f? ) ? Λ for all ? (this is the space O(λ,k) C ?ω (Λ) below).

SELBERG ZETA FUNCTION

633

1.4. The main result. In the present paper we prove the conjecture of Patterson for Γ ? SO(1, n)0 a convex cocompact, non-cocompact and torsion-free subgroup and the trivial representation σ of M . We restrict ourselves to this special case mainly because of the lack of information about the Selberg zeta function in the other cases. Let us ?rst de?ne O(λ,k) C ?ω (Λ). The group G acts on the geodesic boundary ?X ? = S n?1 by means of conformal automorphisms. Let ?X = ? ∪ Λ be the decomposition of ?X into the limit set Λ and the domain of discontinuity ?. For any λ ∈ C let Vλ be the representation of P on C given by man → ρ ? λ a := e(ρ?λ) log(a) . Let V (λ) := G ×P Vλ be the associated homogeneous ? ?X is the complexi?ed bundle of line bundle. Note that V (?ρ) ? = Λn?1 TC n?1?2λ volume forms. Moreover, V (λ) ? = (Λn?1 T ? ?X ) 2(n?1) . If we choose a nowhereC

vanishing volume form vol on ?X , then vol 2(n?1) is a section trivializing V (λ). Sections of V (λ) can thus be viewed as functions which transform under G according to a conformal weight related to λ. The union λ∈C V (λ) → ?X has the structure of a holomorphic family of line bundles. ?Using the nowhere-vanishing volume form vol we de?ne isomorphisms vol? n?1 : V (λ) ? = V (λ + ?). Hence we can identify the space of sections of V (λ) of a given regularity with the corresponding space of sections of a ?xed bundle, e.g., of the trivial one V (ρ). This allows us to speak of holomorphic families of sections or homomorphisms. By π λ (g) : C ?ω (?X, V (λ)) → C ?ω (?X, V (λ)), g ∈ G, we denote the representation of G on the space of hyperfunction sections of V (λ). As a topological vector space C ?ω (?X, V (λ)) is the space of continuous linear functionals on C ω (?X, V (?λ)). If g ∈ G is ?xed, then π λ (g) depends holomorphically on λ. Since Λ is Γ-invariant the space of hyperfunctions C ?ω (Λ, V (λ)) ? C ?ω (?X, V (λ)) with support in Λ carries a representation of Γ induced by π λ . Let Oλ C ?ω (Λ) denote the space of germs at λ of holomorphic families of sections C ? ? → f? ∈ C ?ω (?X, V (?)) with supp(f? ) ? Λ. The representation of Γ on that space is given by (π (g)f )? := π ? (g)f? , g ∈ Γ. k k If Lk λ denotes the multiplication operator Lλ : f? → (? ? λ) f? , k ∈ N, then we have a short exact sequence

λ 0 → Oλ C ?ω (Λ) → Oλ C ?ω (Λ) → O(λ,k) C ?ω (Λ) → 0

n?1?2λ

Lk

of Γ-modules de?ning O(λ,k) C ?ω (Λ). Note that O(λ,1) C ?ω (Λ) ? = C ?ω (Λ, V (λ)). Now we can formulate the main theorem of our paper.

634

ULRICH BUNKE AND MARTIN OLBRICH

Theorem 1.3. For any λ ∈ C there is k(λ) ∈ N0 such that the following assertions hold : (i) dim H ? (Γ, O(λ,k) C ?ω (Λ)) < ∞ for all k, dim H ? (Γ, Oλ C ?ω (Λ)) < ∞. (ii) χ(Γ, O(λ,k) C ?ω (Λ)) = 0 for all k. (iii) If k ≥ k(λ), then dim H ? (Γ, O(λ,k+1) C ?ω (Λ)) = dim H ? (Γ, O(λ,k) C ?ω (Λ)). (iv) If k ≥ k(λ), then the order of the Selberg zeta function at λ is given by (3) (4) ords=λ ZS (s) = = ?χ1 (Γ, O(λ,k) C ?ω (Λ)) , ?χ(Γ, Oλ C ?ω (Λ))

where for any Γ-module V with dim H ? (Γ, V ) < ∞ its ?rst derived Euler characteristic χ1 (Γ, V ) is de?ned by

n

χ1 (Γ, V ) :=

p=1

p(?1)p dim H p (Γ, V ) .

It will be shown in Proposition 4.19 that one can take k(λ) := Ord?=λ ext? + ε, where ε = 0 if λ ∈ ?N0 ? ρ, and ? = 1 otherwise, and also where ext? is the extension map explained in the next subsection. In contrast to ord, Ord?=λ denotes the (positive) order of a pole at ? = λ, if there is one, and it is zero otherwise. For generic Γ and most λ one expects k(λ) ≤ 1. For those λ one can replace O(λ,k) C ?ω (Λ) by C ?ω (Λ, V (λ)) and probably also by C ?∞ (Λ, V (λ)), the space appearing in Patterson’s original conjecture. In [38] the order of ZS (s) at s = 0 was not given explicitly. As a corollary of our computations we obtain: Corollary 1.4. ords=0 ZS (s) = dim Γ C ?ω (Λ, V (0)) = dim ker(S0 + id) , where S0 is the normalized scattering matrix (introduced in Section 2) at zero. The proof of Theorem 1.3 consists of three steps. The ?rst step which occupies most of the paper is an explicit computation of the cohomology groups H ? (Γ, O(λ,k) C ?ω (Λ)), H ? (Γ, Oλ C ?ω (Λ)). One of our main tools in these computations is the extension map which will be explained in the next subsection. The ?nal results which are of interest in their own right, and much more detailed than needed for the proof of Theorem 1.3, are stated in subsections 4.3 and 4.4. They are generalizations of our results for the two-dimensional case [6].

SELBERG ZETA FUNCTION

635

In a second step we compare the result of this computation with the spectral description of the singularities of ZS given by Patterson-Perry [38] in case n ≡ 0(2), δΓ < 0. Finally we employ the embedding trick in order to drop this assumption. The second and third steps are performed in Section 5. In order to compute H ? (Γ, Oλ C ?ω (Λ)) we use suitable acyclic resolutions by Γ-modules formed by germs of holomorphic families of hyperfunctions on ?X and ?. The proof of exactness and acyclicity of these resolutions is quite involved and uses some hyperfunction theory and analysis on the symmetric space X . It turned out that we need facts which hold true in much more general situations but have not been considered in the literature so far (up to our knowledge). This accounts for the length of subsections 4.1 and 4.2. 1.5. The extension map. The present paper has a close companion [8] in which we consider the decomposition of the right regular representation of G on L2 (Γ\G) into irreducible unitary representations in the case that Γ is a convex cocompact subgroup of a simple Lie group of real rank one. The main ingredient of both papers is the extension map extλ . Consider any Γ-invariant hyperfunction section f ∈ Γ C ?ω (?, V (λ)). By the ?abbiness of the sheaf of hyperfunction sections it can be extended across ? ∈ C ?ω (?X, V (λ)) which restricts Λ; i.e., there is a hyperfunction section f ? is not Γ-invariant. Our extension map solves the problem to f . In general f of ?nding a Γ-invariant extension extλ (f ). It turns out that the invariant extension exists and is unique for generic λ. Now the maps extλ form in fact a meromorphic family of maps with ?nitedimensional singularities. The highest singular part of ext at the poles of ext gives invariant hyperfunction (in fact distribution) sections of V (λ) with support on Λ. This can be considered as a generalization of the construction of the Patterson-Sullivan measure which is given by the residue of ext at λ = δΓ . In Section 3 we employ a version of Green’s formula in order to get a hold on the spaces of invariant hyperfunction sections with support on Λ. In particular, it follows that there is a discrete set of λ ∈ C, where f ∈ Γ C ?ω (?, V (λ)) has to satisfy a ?nite number of nontrivial linear conditions in order to be invariantly extendable. To ?nd a Γ-invariant extension of f is a cohomological problem and H 1 (Γ, C ?ω (Λ, V (λ)) essentially appears as its obstruction group. This is the basic observation which enables us to compute these cohomology groups. In order to provide a feeling for the extension problem let us discuss a toy example. We consider the extension of hyperfunctions f on R \ {0}, which λ transform as f (rx) = r λ f (x) for all r ∈ R? + . Let fλ be given by fλ (x) := |x| . For Re(λ) > ?1 its invariant extension as a distribution (and hence as a

636

ULRICH BUNKE AND MARTIN OLBRICH

hyperfunction) to R is just given by extλ (fλ ), φ :=

R

φ(x)|x|λ dx ,

∞ φ ∈ Cc (R) .

It is well-known that extλ (fλ ) has a meromorphic continuation to all of C with poles at negative integers. The residues of the continuation at these points are proportional to derivatives of delta distributions located at {0}. The construction of the meromorphic continuation of the extension map is closely related to the meromorphic continuation of the scattering matrix Sλ : Γ C ?ω (?, V (λ)) → Γ C ?ω (?, V (?λ)) . If Γ is the trivial subgroup, then ? = ?X and Sλ coincides with the (normalized) Knapp-Stein intertwining operator Jλ : C ?ω (?X, V (λ)) → C ?ω (?X, V (?λ)) . The operators Jλ form a meromorphic family and are well-studied in representation theory. In this paper and in [8] we approach the scattering matrix Sλ starting from Jλ and using the basic identity where resλ denotes the restriction resλ : Γ C ?ω (?X, V (λ)) → Γ C ?ω (?, V (λ)). In the literature the scattering matrix is usually considered as a certain pseudodi?erential operator which can be applied to smooth, resp. distribution sections. A meromorphic continuation of Sλ was obtained in [35] for surfaces, and in [40], [30] in general. In [8] we develop the theory of the scattering matrix and the extension map in a smooth/distribution framework for general rank-one spaces and arbitrary σ . The main point in the present paper is the transition to the real analytic/hyperfunction framework. The main di?erence from most previous papers is that our primary analysis concerns objects on the boundary ?X . The spectral theory of ?Y is only needed in some very weak form. One the other hand one can deduce the spectral decomposition, the meromorphic continuation of Eisenstein series and the properties of the resolvent kernel from the theory on the boundary. To illustrate this consider, e.g., the Eisenstein series. Let Pλ : C ?ω (?X, V (λ)) → C ∞ (X ) denote the Poisson transform. For b ∈ ? we de?ne fb ∈ Γ C ?ω (?, V (λ)) by fb := γ ∈Γ π λ (γ )(δb vol? ), where δb ∈ C ?ω (?, V (?ρ)) is the delta distribu?1+2λ tion located at b and ? := ? n 2(n?1) . Then the Eisenstein series can be written as Eλ (x, b) := (Pλ ? extλ (fb ))(x) . These applications will be contained in [8] and its continuations. Acknowledgement. We thank S. Patterson and P. Perry for keeping us informed about the progress of [38]. Moreover we thank A. Juhl for pointing out Sλ = res?λ ? Jλ ? extλ ,

SELBERG ZETA FUNCTION

637

some wrong arguments in a previous version of this paper and for further useful remarks. This work was partially supported by the Sonderforschungsbereich 288, Di?erentialgeometrie und Quantenphysik. 2. Restriction, extension, and the scattering matrix 2.1. Basic notions. The sheaf of hyperfunction sections of a real analytic vector bundle over a real analytic manifold is ?abby. Thus the following sequence of Γ-modules

? C ?ω (?, V (λ)) → 0 0 → C ?ω (Λ, V (λ)) → C ?ω (?X, V (λ)) →

res

is exact, where res? is the restriction of sections to ?. Let VB (λ) := Γ\V (λ)|? . If we identify Γ C ?ω (?, V (λ)) ? = C ?ω (B, VB (λ)), then res? induces a map resλ : Γ C ?ω (?X, V (λ)) → C ?ω (B, VB (λ)). Here Γ C ?ω (., V (λ)) denotes the subspace of Γ-invariant sections. The main topic of this section is the construction of a meromorphic family of maps extλ : C ?ω (B, VB (λ)) → Γ C ?ω (?X, V (λ)) which are right inverse to resλ . The poles of extλ will correspond exactly to those points λ ∈ C where resλ fails to be an isomorphism. The strategy of the construction of extλ is the following. We ?rst con?λ struct extλ for Re(λ) > δΓ . Then we introduce the scattering matrix S : C ?ω (B, VB (λ)) → C ?ω (B, VB (?λ)) by (5) ?λ := res?λ ? J ?λ ? extλ , S ?λ : C ?ω (?X, V (λ)) → C ?ω (?X, V (?λ)) is the Knapp-Stein intertwinwhere J ing operator (see [26]) which we will introduce below. Assuming for a moment ?λ using results of that δΓ < 0 we obtain a meromorphic continuation of S Patterson [34] and [8]. Then we construct the meromorphic continuation of extλ by (6) extλ := J?λ ? ext?λ ? Sλ , Re(λ) < 0 ,

where J?λ (Sλ ) is the normalized intertwining operator (scattering matrix). If δΓ ≥ 0, then we employ the embedding SO(1, n)0 ?→ SO(1, n + m)0 , m su?ciently large, in order to reduce to the case δΓ < 0. 2.2. Holomorphic functions to topological vector spaces. In order to carry out the program sketched above we need to consider holomorphic families of vectors in topological vector spaces. We will also consider holomorphic families of continuous linear maps between such vector spaces. Therefore we collect some preparatory material of a technical nature. All topological vector spaces appearing in this paper are Hausdor? and complete.

638

ULRICH BUNKE AND MARTIN OLBRICH

A holomorphic family of vectors in a locally convex topological vector space F de?ned on U ? C is by de?nition a continuous function from U to F which is weakly holomorphic. Using Cauchy’s integral formula one can show an equivalent characterization of holomorphic families. A map f : U → F is holomorphic, if and only if for any z0 ∈ U there is a neighbourhood z0 ∈ V ? U k and a sequence {fk ∈ F}k∈N0 such that for z ∈ V the sum ∞ k =0 fk (z ? z0 ) converges and is equal to f (z ). In order to speak of holomorphic families of homomorphisms from F to G , where G is another locally convex topological vector space we equip Hom(F , G ) with the topology of uniform convergence on bounded sets. Let f : U \ {z0 } → Hom(F , G ) be holomorphic and f (z ) = ∞ k =?N fk (z ? k z0 ) for all z = z0 close to z0 . Then we say that f is meromorphic and has a pole of order N at z0 . If fk , k = ?N, . . . , ?1, are ?nite-dimensional, then, by de?nition, f has a ?nite-dimensional singularity. Holomorphy of a map f : U → Hom(F , G ) can be characterized in the following weak form. Let G ′ denote the dual space of G with its strong topology. We call a subset A ? F × G ′ su?ciently large if for B ∈ Hom(F , G ) the condition φ, Bψ = 0, for all (ψ, φ) ∈ A, implies B = 0. Lemma 2.1. The following assertions are equivalent : (i) f : U → Hom(F , G ) is holomorphic. (ii) f : U → Hom(F , G ) is continuous, and there is a su?ciently large set A ? F × G ′ such that for all (ψ, φ) ∈ A the function U ? z → φ, f (z )ψ is holomorphic. Proof. It is obvious that (i) implies (ii). We show that (i) follows from (ii). It is easy to see that f is holomorphic at z if and only if f (z ′ ) = 1 2π? f (x) dx x ? z′

for z ′ close to z , where the path of integration is a small circle surrounding z counterclockwise. Let f satisfy (ii). Then we form d(z ′ ) := f (z ′ ) ? 1 2π? f (x) dx . x ? z′

We must show that d = 0. It su?ces to show that for all (ψ, φ) ∈ A we have φ, d(z ′ )ψ = 0. But this follows from (ii) and Cauchy’s integral formula. Lemma 2.2. Let fi : U → Hom(F , G ) be a sequence of holomorphic maps. Moreover let f : V → Hom(F , G ) be continuous such that for a su?ciently large set A ? F × G ′ the functions φ, fi ψ converge locally uniformly in U to φ, f ψ . Then f is holomorphic, too.

SELBERG ZETA FUNCTION

639

Proof. Since the holomorphic functions φ, fi ψ converge locally uniformly to φ, f ψ we conclude that the latter function is holomorphic, too. The lemma is now a consequence of Lemma 2.1. Lemma 2.3. Let f : U → Hom(F , G ) be continuous. Then the adjoint f ′ : U → Hom(G ′ , F ′ ) is continuous. If f is holomorphic, then so is f ′ . Proof. We ?rst show that the adjoint f ′ : U → Hom(G ′ , F ′ ) is continuous at z0 ∈ U . Let B ? G ′ be a bounded set. Let q be any continuous seminorm on F ′ . Then we have to show that for any ? > 0 there exists δ > 0 such that if |z ? z0 | < δ, then supφ∈B q (f ′ (z )φ ? f ′ (z0 )φ) < ?. The strong topology of F ′ is generated by the seminorms qD associated to bounded subsets D ? F , where qD (ψ ) := supκ∈D | ψ, κ |. We have

φ∈B

sup qD (f ′ (z )φ ? f ′ (z0 )φ) = = =

sup

φ∈B,κ∈D

| f ′ (z )φ ? f ′ (z0 )φ, κ |

sup

φ∈B,κ∈D κ∈D

| φ, f (z )κ ? f (z0 )κ |

sup qB (f (z )κ ? f (z0 )κ) .

Here qB is a continuous seminorm on the bidual G ′′ . Since the embedding G ?→ G ′′ is continuous and f is continuous at z0 we can ?nd δ > 0 for any ? > 0 as required. If f is holomorphic, then holomorphy of f ′ follows from Lemma 2.1 when we take the su?ciently large set F ×G ′ , and use the fact that φ, f ψ = f ′ φ, ψ is holomorphic for all ψ ∈ F , φ ∈ G ′ . A locally convex vector space is called Montel if its closed bounded subsets are compact. Lemma 2.4. Let F , G , H be locally convex topological vector spaces and assume that F is a Montel space. If f : U → Hom(F , G ) and f1 : U → Hom(G , H) are continuous, then the composition f1 ? f : V → Hom(F , H) is continuous. If f and f1 are holomorphic, then so is the composition f1 ? f . Proof. We ?rst prove continuity of the composition f1 ? f at z0 ∈ U . It is here where we need the assumption that F is Montel. Let B ? F be a bounded set and s be a seminorm of H. We have to show that for all ? > 0 there exists δ > 0 such that if |z ? z0 | < δ, then supφ∈B s((f1 ? f )(z )(φ) ? (f1 ? f )(z0 )(φ)) < ?. Let W ? U be a compact neighbourhood of z0 . The map F : W × B → G given by F (v, φ) := f (v )(φ) is continuous. Since F is Montel, any bounded set B is precompact. Thus the image B1 := F (W × B ) of the compact set W × B is precompact, too. In particular, B1 is bounded. Since f1 (z0 ) is continuous there is a seminorm t of G such that s(f1 (z0 )(ψ )) < t(ψ ), for all ψ ∈ G . Now let V ? W be a neighbourhood of z0 so small that t(f (z )(φ) ? f (z0 )(φ)) < ?/2,

640

ULRICH BUNKE AND MARTIN OLBRICH

for all z ∈ V , for all φ ∈ B , and s(f1 (z )(ψ ) ? f1 (z0 )(ψ )) < ?/2, for all ψ ∈ B1 , for all z ∈ V . Then for all z ∈ V , φ ∈ B , s(f1 (z ) ? f (z )(φ) ? f1 (z0 ) ? f (z0 )(φ)) ≤ s([f1 (z ) ? f1 (z0 )]f (z )(φ)) < ?. +s(f1 (z0 )[f (z )(φ) ? f (z0 )(φ)])

Thus we can ?nd δ > 0 for any ? > 0 as required. This proves continuity of the composition f1 ? f . We now show that this composition is holomorphic if f, f1 are so. We have f1 (x) ? f (x) 1 1 f1 (x) ? f (y ) dx = dy dx . 2π? x?z (2π?)2 (x ? z )(y ? x)

If we restrict this equation to a bounded set B ? F , we see as above that there (y ) exists some bounded set B1 ? G such that (x?f z )(y ?x) (B ) ? B1 for all y, x in the domain of integration. Hence we can apply Fubini’s theorem to the double integral and obtain 1 2π? f1 (x) ? f (x) dx x?z = = = 1 f1 (x) ? f (y ) dx dy 2 (2π?) (x ? z )(y ? x) f1 (z ) ? f (y ) 1 dy 2π? y?z (f1 ? f )(z ) .

This shows that f1 ? f is holomorphic. Consider a real analytic vector bundle over a closed real analytic manifold. Then the spaces of real analytic, smooth, distribution, and hyperfunction sections of the bundle equipped with their natural locally convex topologies are Montel spaces. 2.3. The push-down. In the present subsection we de?ne a push-down map π?,?λ : C ? (?X, V (?λ)) → C ? (B, VB (?λ)), ? ∈ {ω, ∞}, Re(λ) > δΓ .

The extension extλ will then appear as its adjoint. Using the identi?cation C ? (B, VB (?λ)) ? = Γ C ? (?, V (?λ)) we want to de?ne π?,?λ by (7) π?,?λ (f )(b) = (π ?λ (g)f )(b),

g ∈Γ

b ∈ ?,

f ∈ C ? (?X, V (?λ))

provided the sum converges. In order to prove the convergence for Re(λ) > δΓ we need the following two geometric lemmas. We adopt the following conventions about the notation for points of X and ?X . A point x ∈ ?X can equivalently be denoted by a subset kM ? K

SELBERG ZETA FUNCTION

641

or gP ? G representing this point in ?X = K/M or ?X = G/P . If F ? ?X , then F M := kM ∈F kM ? K . Analogously, we can denote a point b ∈ X by a set gK ? G, where gK represents b in X = G/K . Adjoining the boundary at in?nity we can consider X ∪ ?X as a compact manifold with boundary carrying a smooth action of G. Let Γ ? G be a torsion-free convex cocompact subgroup. An equivalent characterization of being convex cocompact is that Γ acts freely and cocompactly on X ∪ ?. Lemma 2.5. If F ? ? is compact, then ?(Γ ∩ (F M )A+ K ) < ∞. Proof. Note that (F M )A+ K ∪ F ? X ∪ ? is compact. Thus its intersection with the orbit ΓK of the origin of X is ?nite. Using the Iwasawa decomposition G = KAN we write g ∈ G as g = κ(g)a(g)n(g) ∈ KAN . The Iwasawa decomposition and, in particular, the maps g → κ(g), g → a(g), g → n(g) are real analytic and extend to a complex neighbourhood of G in its complexi?cation GC . Let KC , AC be the complexi?cations of K, A. We identify AC with the multiplicative group C? such that A+ corresponds to [1, ∞) ? C? . Any g ∈ G has a Cartan decomposition g = hag h′ ∈ KA+ K , where ag is uniquely determined. The next lemma gives some control on the complex extension of the Iwasawa decomposition. Lemma 2.6. Let k0 M ∈ ?X . For any compact W ? (?X \ k0 M )M and complex neighbourhood SC ? KC of K there are a complex neighbourhood UC ? KC of k0 M and constants c > 0, C < ∞, such that for all g = hag h′ ∈ W A+ K the components κ(g?1 k) and a(g?1 k) extend holomorphically to k ∈ UC , and for all k ∈ UC (8) (9) κ(g?1 k) ∈ SC . cag ≤ |a(g?1 k)| ≤ Cag ,

Proof. The set W ?1 k0 M is compact and disjoint from M . Let w ∈ NK (M ) ? := θ (n), represent the nontrivial element of the Weyl group of (g, a). Set n ? where θ is the Cartan involution of G ?xing K , and de?ne N := exp(? n). By ? P ∪ P we have K = wκ(N ? )M ∪ M . Thus the Bruhat decomposition G = wN ? such that W ?1 k0 M ? wκ(V )M . By enlarging V there is a compact V ? N ? by (a, n we can assume that V is A+ -invariant, where A acts on N ? ) → an ? a?1 . ?C of There exists a complex compact A+ -invariant neighbourhood VC ? N V such that κ(? n), a(? n), n(? n) extend to VC holomorphically. Moreover, there 1 ? K of k M such that w?1 W ?1 U 1 M ? exists a complex neighbourhood UC 0 C C 1 κ(VC )M . Let k ∈ UC and g = hag h′ ∈ W A+ K . Then h?1 k = wκ(? n)m for 1 by V × M . Furthermore, n ? ∈ VC , m ∈ MC ; i.e., we parametrize h?1 UC C C

642

ULRICH BUNKE AND MARTIN OLBRICH

a(g?1 k) = = = = =

1 ?1 a(h′?1 a? g h k) 1 a(a? n)m) g wκ(?

a(ag κ(? n)) a(ag n ? n(? n)?1 a(? n)?1 )

1 a(ag n ? a? n)?1 ag . g )a(?

?1 1 1 Now ag n ? a? g ∈ VC . Thus a(g k ) extends holomorphically to k ∈ UC . Set

c := C :=

n ? ∈VC

inf |a(? n)| inf |a(? n)?1 |

n ? ∈VC n ? ∈VC

n ? ∈VC

sup |a(? n)| sup |a(? n)?1 | .

Since VC is compact we have 0 < c ≤ C < ∞. Then cag ≤ |a(g?1 k)| ≤ Cag . Now considering κ we have κ(g?1 k)

1 ?1 = κ(h′?1 a? g h k) 1 = h′?1 κ(a? n))m g wκ(?

= h′?1 wκ(ag n ? n(? n)?1 a(? n)?1 )m

1 = h′?1 wκ(ag n ? a? g )m . 1 ?1 1 Since ag n ? a? g ∈ VC we see that κ(g k ) extends holomorphically to k ∈ UC . ′? 1 If we take VC small enough we can also satisfy h wκ(VC ) ? SC . Thus for a 1 we have κ(g ?1 k ) ∈ S smaller open subset UC ? UC C for all g ∈ W A+ K and k ∈ UC .

Lemma 2.7. If Re(λ) > δΓ , then the sum (7) converges and de?nes a holomorphic family of continuous maps π?,?λ : C ? (?X, V (?λ)) → C ? (B, VB (?λ)), ? ∈ {∞, ω } .

Proof. In case ? = ∞ the lemma was proved in [8]. Thus we assume ? = ω . First we recall the de?nition of the topology on the spaces of real analytic sections of real analytic vector bundles. We describe C ω (?X ) as a direct limit of Banach spaces. Let SC ? KC be a compact right M -invariant complex neighbourhood of K . Let H(SC ) denote the Banach space of bounded holomorphic functions on SC equipped with the norm f = sup |f (k)|, f ∈ H(SC ) .

k ∈SC

SELBERG ZETA FUNCTION

643

′ ? S , then we have a continuous restriction H(S ) ? ′ If SC C C → H(SC ). Then

C ω (K ) = lim H(SC )

?→

as a topological vector space, where the limit is taken over all compact right M -invariant complex neighbourhoods of K . The compact group M acts continuously on H(SC ), and this action is compatible with the restriction to smaller ′ . Thus the natural right action of M on C ω (K ) is continuous. We obtain SC C ω (?X ) as the closed subspace of right M -invariants in C ω (K ). Using the K -invariant volume form on ?X we identify C ω (?X, V (λ)) ? = C ω (?X ). ω Next we describe the topological structure of C (B, VB (λ)). Let {Fα } be a ?nite cover of B by compact neighbourhoods which have di?eomorphic lifts ?α ? ?. Using again the K -invariant volume form we identify C ω (F ?α , V (λ)) F M with the direct limit of the Banach spaces H(UC,α ) of M -invariant bounded holomorphic functions on UC,α where UC,α ? KC are M -invariant compact ?α M . Any section f ∈ C ω (B, VB (λ)) de?nes seccomplex neighbourhoods of F ω ? tions fα ∈ C (Fα , V (λ)) by restricting the lift of f . The correspondence f → α fα de?nes an embedding I : C ω (B, VB (λ)) ?→ ?α , V (λ)) C ω (F

α

as a closed subspace and, hence, a topology on C ω (B, VB (λ)). The space C ω (B, VB (λ)) depends on λ. Since Γ acts on ? preserving the orientation the manifold B is orientable, and we can ?nd an analytic realvalued nowhere-vanishing volume form volB ∈ C ω (B, VB (?ρ)). Multiplcation by suitable complex powers of volB provides isomorphisms between any two bundles VB (λ1 ) and VB (λ2 ). Thus we may identify C ω (B, VB (λ)) with the ?xed space C ω (B, VB (ρ)) = C ω (B ). Using this identi?cation we can speak of continuous or holomorphic families of continuous homomorphisms to or from C ω (B, VB (λ)). Let G be any locally convex topological vector space, U ? C be open, and U ? λ → φλ ∈ Hom(G , C ω (B, VB (λ)) be a family of continuous maps. This family depends continuously (holomorphically) on λ, if and only if the composition I ? φλ does. ? ? ? be its lift. Let F ? ? U1 ? ? be an open neighLet F ∈ {Fα } and F ? bourhood of F and W := (?X \ U1 )M . By Lemma 2.6 we can ?nd for any complex M -invariant neighbourhood SC ? KC of K a complex neighbour? M such that a(g?1 k), κ(g?1 k) extend to k ∈ UC for all hood UC ? KC of F g ∈ W A+ K and κ(g?1 k) ∈ SC . This allows us to de?ne the map resUC ? π λ (g) : H(SC ) → H(UC ) for all g ∈ W A+ K . Here (π λ (g)f )(k) = a(g?1 k)λ?ρ f (κ(g?1 k)), f ∈ H(SC ), k ∈ UC .

644

ULRICH BUNKE AND MARTIN OLBRICH

In order to estimate the norm of resUC ? π λ (g) we compute for f ∈ H(SC ) using (8), resUC ? π λ (g)f = = ≤ =

k ∈UC k ∈UC

sup |resUC ? π λ (g)(f )(k)| sup |a(g?1 k)λ?ρ f (κ(g?1 k))|

k ∈SC

Reλ?ρ Cag sup |f (k)| Reλ?ρ Cag

f ,

where C is independent of k, g, f . By Lemma 2.5 we have ?(Γ \ Γ ∩ W A+K ) < ∞. Thus for almost all g ∈ Γ Reλ?ρ . Fix ? > 0. Then by the de?nition of δ we have resUC ? π (g) ≤ Cag Γ the sum resUC ? π λ (g)f ∈ H(UC ) H(SC )M ? f →

g ∈Γ∩W A+ K

converges uniformly for Re(λ) ∈ (?∞, ?δΓ ? ?), and on the unit ball of the Banach space H(SC )M . Going over to the direct limits we conclude that λ ΠF ? ? π (g ) is a convergent sum of holomorphic maps g ∈Γ resF λ := which is uniformly convergent on bounded subsets of C ω (?X, V (λ)). Thus ΠF λ is holomorphic with respect to λ by Lemma 2.2. In order to ?nish the proof of α Lemma 2.7 observe that I ? π?,λ = α ΠF λ . For any λ ∈ C we have V (λ) ? V (?λ) = Λn?1 T ? ?X ? C. Integration induces a G-invariant pairing of sections of V (λ) with sections of V (?λ). Therefore we de?ne C ?ω (?X, V (λ)) := C ω (?X, V (?λ))′ . Similarly we proceed with VB (λ) and distribution sections of V (λ) and VB (λ), respectively. De?nition 2.8. For Re(λ) > δΓ and ? ∈ {ω, ∞} de?ne the extension map ′ extλ : C ?? (B, VB (λ)) → C ?? (?X, V (λ)) by extλ := π? , ?λ . 2.4. Elementary properties of extλ . Lemma 2.9. extλ ∈ Hom(C ?? (B, VB (λ)), C ?? (?X, V (λ))) depends holomorphically on λ. Proof. This follows from Lemmas 2.7 and 2.3. Lemma 2.10. Let Re(λ) > δΓ and ? ∈ {ω, ∞}. Then the extension map extλ has values in Γ C ?? (?X, V (λ)). For Re(λ) > δΓ and ? ∈ {ω, ∞} the map

λ ω ω ? {Re(λ) < ?δΓ ? ?} ? λ → resF ? ? π (g ) ∈ Hom(C (?X, V (λ)), C (F , V (λ))) ,

SELBERG ZETA FUNCTION

645

Proof. This follows immediately from the equation π?,?λ (π ?λ (g)f ) = π?,?λ (f ), for all f ∈ C ? (?X, V (?λ)). Next we want to show that extλ is right inverse to resλ . For distributions this was shown in [8]. For hyperfunctions we want to argue by continuity. Therefore we must show that resλ is continuous. This is not straightforward since we have de?ned resλ as the composition of res? and an identi?cation, and there is no natural topology on C ?ω (?, V (λ)). Lemma 2.11. is continuous. Let λ ∈ C. Then the restriction map resλ : Γ C ?ω (?X, V (λ)) → C ?ω (B, VB (λ))

Proof. We begin by giving a description of resλ which is more appropriate for the present purpose. It is based on an alternative de?nition of the topological vector space C ?ω (?X, V (λ)). Let {Fα } be a ?nite cover of B by compact neighbourhoods which have ?α ? ?. We consider the ?nite set L := {g ∈ Γ | ?α, β such di?eomorphic lifts F ? ?α ∩ F ?β = ?}. Set W := clo(?X \ that gF g ∈L,α g Fα ). Then any hyperfunction ? ω f ∈ C (?X, V (λ)) can be represented as a sum of hyperfunctions fg,α , fW ∈ ?α , and supp(fW ) ? W . C ?ω (?X, V (λ)), g ∈ L, where supp(fg,α ) ? gF If f is Γ-invariant, then by lifting a corresponding decomposition of resλ (f ) on B to ? we can choose the hyperfunctions fg,α such that (10) fg,α = πλ (g)f1,α , for all α, g ∈ L . This choice made, we have resλ (f ) = α f1,α , where we view f1,α as an element of C ?ω (B, VB (λ)) with support in Fα . We now argue that resλ is continuous. Consider the Fr? echet spaces ?ω ? ω ?α , V (λ)) ⊕ C (W, V (λ)) , C (gF G :=

g ∈L,α L

Here for any closed set A ? ?X the space C ?ω (A, V (λ)) is the space of hyperfunction sections of V (λ) with support on A, i.e. the topological dual of the space of germs at A of real analytic sections of V (?λ). By the above we have a surjective continuous map φ : G → C ?ω (?X, V (λ)) with the property that Γ C ?ω (?X, V (λ)) ? φ(L G ). There is a continuous map res ? : G → C ?ω (B, VB (λ)) given by res( ? f) L := α f1,α . For f ∈ G we have res( ? f )|int(Fα ) = φ(f )|int(F . Thus res ? ?α ) | ker φ∩L G = 0, and res ? factorizes over the Fr? echet space F de?ned by F := [φ?1 (Γ C ?ω (?X, V (λ))) ∩ L G ]/[ker(φ) ∩ L G ] . ? : F → Γ C ?ω (?X, V (λ)), and resλ = res ??1 . φ induces an isomorphism φ ? ?φ Hence resλ is continuous.

G

:=

{f ∈ G | f satis?es (10)} .

646

ULRICH BUNKE AND MARTIN OLBRICH

Lemma 2.12. For Re(λ) > δΓ and ? ∈ {ω, ∞} the map extλ satis?es resλ ? extλ = id . Proof. For ? = ∞ this is as shown in [8, Lemma 4.5]. Since C ?∞(B,VB (λ)) is dense in C ?ω (B, VB (λ)), and resλ ? extλ is continuous, the assertion for ? = ω follows. If Γ C ?ω (Λ, V (λ)) = 0, then resλ is injective. If in addition Re(λ) > δΓ , then we have extλ ? resλ = id. In order to apply this observation we need vanishing results for Γ C ?ω (Λ, V (λ)). Lemma 2.13. If Re(λ) > 0 and Im(λ) = 0, then Γ C ?ω (Λ, V (λ)) = 0.

Remark. We will obtain much stronger vanishing results later. Proof. The proof is based on the fact that the Laplace-Beltrami operator ?Y is self-adjoint. If 0 = φ ∈ Γ C ?ω (Λ, V (λ)), then, using the Poisson transform Pλ , we will construct a nontrivial L2 -eigenfunction Pλ φ of ?Y to an eigenvalue ? ∈ R, obtaining a contradiction. We now explain the details. Let g = κ(g)a(g)n(g) ∈ KAN be the Iwasawa decomposition of g ∈ G. De?nition 2.14. The Poisson transform. Pλ : C ?ω (?X, V (λ)) → C ∞ (X ) is de?ned by (Pλ φ)(gK ) := a(g?1 k)?(λ+ρ) φ(k)dk .

K

Here φ ∈ C ?ω (?X, V (λ)) is viewed as a “function” on G with values in Vλ ? = C, and the integral is a formal notation meaning that the analytic functional φ has to be applied to the analytic integral kernel. The Poisson transform Pλ is continuous, G-equivariant, and (?X ? ρ2 + λ2 )Pλ φ = 0. It is injective whenever λ ∈ ?ρ ? N0 . In this case Pλ provides a topological isomorphism between C ?ω (?X, V (λ)) and ker (?X ? ρ2 + λ2 ) : C ∞ (X ) → C ∞ (X ) .

For all these facts see [23], [18], or [45]. Let V ? ?X and U ? X be such that clo(U ) ∩ V = ?, where we take the closure of U in X ∪ ?X . It is not di?cult to show (see Lemma 3.2, (ii) below) that for Re(λ) > 0 and φ ∈ C ?ω (?X, V (λ)), supp φ ? V , the restriction of the Poisson transform Pλ φ to U belongs to L2 (U ). If φ ∈ Γ C ?ω (Λ, V (λ)), then Pλ φ is Γ-invariant, and therefore it descends to an eigenfunction of ?Y in L2 (Y ).

SELBERG ZETA FUNCTION

647

Since Y is complete, the operator ?Y is self-adjoint with domain {f ∈ L2 (Y )|?Y f ∈ L2 (Y )}. In particular ?Y cannot have nontrivial eigenvectors in L2 (Y ) to eigenvalues with nontrivial imaginary part. Since Im(λ2 ) = 0 we conclude that Pλ φ = 0 and hence φ = 0 by injectivity of the Poisson transform. 2.5 The scattering matrix. ?λ de?ned by (5). Our In this subsection we study the scattering matrix S investigation is based on a detailed knowledge on the Knapp-Stein intertwining ?λ ([26], [50, Ch.10]). In order to ?x our normalization conventions operators J ?λ for Re(λ) < 0. let us ?rst give a de?nition of J De?nition 2.15. Consider f ∈ C ∞ (?X, V (λ)) as a right P -equivariant ?λ : C ∞ (?X, V (λ)) → C ∞ (?X, V (?λ)) function on G with values in Vλ . Then J is de?ned by the convergent integral ?λ f )(g) := (J

? N

f (gwn ? )dn ?.

?λ is de?ned by meromorphic continuation For Re(λ) ≥ 0 the operator J ([26], [50, Ch.10]). It extends continuously to C ?? (?X, V (λ)), ? ∈ {∞, ω }, and ?λ acting on C ? (?X, V (λ)). this extension coincides with the adjoint of J ?λ is meromorphic in the To be precise at this point we must show that J sense of subsection 2.2. Lemma 2.16. For Re(λ) < 0 and ? ∈ {±ω, ±∞} the intertwining opera?λ ∈ Hom(C ? (?X, V (λ)), tors form a holomorphic family of continuous maps J C ? (?X, V (?λ))). This family admits a meromorphic continuation to all of C having at most ?rst order poles. Proof. For ? = ∞ this is as shown in [8]. The case ? = ?∞ follows by duality from Lemma 2.3. We now show the lemma for ? = ω . It is su?cient to prove the ?rst assertion. The proof of the meromorphic continuation [8] (it is based on equation (17) ) for ? = ∞ applies equally well to ? = ω . Let Xi , i = 1, . . . , dim(k), be an orthonormal base of k. For any multiindex r = (i1 , . . . , idim(k) ) we set

dim(k) dim(k) dim(k)

Xr =

l=1

Xlil , |r | =

il , r ! :=

l=1 l=1

il ! ,

and for f ∈ C ω (K ) we de?ne the seminorm f

r

= sup |f (Xr k)| .

k ∈K

648

ULRICH BUNKE AND MARTIN OLBRICH

For any f ∈ C ω (K ) there exists an R > 0 such that (11) f

R

:=

r

R |r | f r!

r

<∞.

Consider the Banach space HR (K ) := {f ∈ C ω (K )| f R < ∞}. If 0 < R′ < R, then we have an inclusion HR (K ) ?→ HR′ (K ). As a topological vector space, C ω (K ) = lim HR (K ) .

?→

C ω (K )M . (12)

Let λ ∈ C and Re(λ) < 0. As usual, we identify C ω (?X, V (λ)) with We extend f ∈ C ω (K ) to a function fλ on G by fλ (kan) = f (k)aλ?ρ .

If f ∈ C ω (?X, V (λ)), then ?λ (f )(k) := J

? N

fλ (kwn ? )dn ?.

For any λ0 ∈ a? C with Re(λ0 ) < 0 and δ > 0 we can ?nd an ε > 0 such that for |λ ? λ0 | < ε

? N

|a(? n)λ0 ?ρ ? a(? n)λ?ρ |dn ?<δ sup | fλ0 (Xr kwn ? ) ? fλ (Xr kwn ? )dn ?|

holds. We then have ? ? J λ0 f ? Jλ f

r

= ≤ ≤

k ∈K

? N

? k ∈K N

sup f

|f (Xr kwκ(? n))||a(? n)λ0 ?ρ ? a(? n)λ?ρ |dn ? |a(? n)λ0 ?ρ ? a(? n)λ?ρ |dn ?

r r

? N

≤ δ f

.

If f ∈ HR (K ), then we conclude

? ? J λ0 f ? Jλ f

R

≤δ f

R

.

This proves continuity of the family of intertwining operators for ? = ω and Re(λ) < 0. Holomorphy now follows from [50, Lemma 10.1.3], and Lemma 2.1. This proves the lemma for ? = ω . In the case ? = ?ω we argue by duality using Lemma 2.3. There is a meromorphic function P (λ) such that the following functional equation holds: (13) ?λ ? J ??λ = J id . P (λ)

The function P (λ) is called the Plancherel density.

SELBERG ZETA FUNCTION

649

Let 1λ ∈ C ω (?X, V (λ)) be the unique K -invariant section normalized such that 1λ (1) = 1 (when viewed as a function on G with values in Vλ ? = C). De?nition 2.17. The meromorphic function c(λ) is de?ned by ?λ 1λ = c(?λ)1?λ . J We de?ne the normalized intertwining operator by ?λ . Jλ := c(?λ)?1 J By (13), (14) 1 = c(λ)c(?λ) . P (λ) Jλ ? J?λ = id .

This implies the following meromorphic identity: (15) ?λ . UnfortuNow we turn to the investigation of the scattering matrix S nately, we are not able to show directly its meromorphic continuation as an operator on real analytic or hyperfunction sections of VB (λ). In [8] we stud?λ as an operator acting on smooth or distribution ied the scattering matrix S sections of VB (λ). First we recall some of these results. Lemma 2.18. (i) For Re(λ) > δΓ the scattering matrix de?ned by ?λ := res?λ ? J ?λ ? extλ : C ?∞ (B, VB (λ)) → C ?∞ (B, VB (?λ)) S forms a meromorphic family of continuous maps, as does the normalized ?λ . scattering matrix Sλ := c(?λ)?1 S ?λ , Sλ admit meromorphic continuations to all of C. The (ii) The families S ?λ and Sλ in the region Re(λ) < 0 are at most ?nitesingularities of S dimensional. (iii) The following functional equation holds : Sλ ? S?λ = id. (iv) The adjoint ?′ : C ∞ (B, VB (λ)) → C ∞ (B, VB (?λ)) S λ ?λ to C ∞ (B, VB (λ)). In particular, this coincides with the restriction of S restriction de?nes a meromorphic family of continuous maps ?λ : C ∞ (B, VB (λ)) → C ∞ (B, VB (?λ)) . S (v) The extension map extλ : C ?∞ (B, VB (λ)) → C ?∞ (?X, V (λ)) has a meromorphic continuation to all of C with at most ?nite-dimensional singularities. Moreover, extλ has values in Γ C ?∞ (?X, V (λ)).

650 (vi) The equations

ULRICH BUNKE AND MARTIN OLBRICH

resλ ? extλ ?λ S extλ

= = =

id, J?λ ? ext?λ ? Sλ ?λ ? extλ , res?λ ? J

extend as meromorphic identities. Proof. All these assertions are shown in [8]. The main point is the meromorphy of the scattering matrix. The remaining assertions are easy consequences. A meromorphic continuation of the scattering matrix was already obtained in several previous papers, e.g. in [34] in case δΓ < 0, and in [40], [30], [31], [38] in the general case. Since these papers use di?erent conventions for continuity we can only deduce meromorphy of matrix coe?cients of the scattering matrix. But we can employ Cauchy’s integral formula in order to obtain the meromorphy of the scattering matrix in the sense of the present paper. Let W ? ?X be a closed set with nonempty interior and consider the space (16) Fλ := {f ∈ C ?ω (?X, V (λ))|f|W ∈ C ω (W, V (λ))} .

We equip Fλ with the weakest topology such that the maps Fλ ?→ C ?ω (?X, V (λ)) and Fλ → C ω (W, V (λ)) are continuous. Using multiplication by suitable powers of a K -invariant volume form we can identify Fλ , λ ∈ C, with the ?xed space F0 . We employ this identi?cation in order to speak of holomorphic families of vectors fλ ∈ Fλ . Let F ? int(W ) be closed. By resF we denote the restriction of sections of V (λ) to F . ?λ : Fλ → C ω (F, V (?λ)) is well Lemma 2.19. The composition resF ? J de?ned and depends meromorphically on λ. Proof. In order to show that the composition is well de?ned we must show ?λ (f ) is real analytic on F . that if f ∈ Fλ , then J As a ?rst step we reduce the proof of the lemma to the case Re(λ) < 0 using the translation principle. In order to write down the appropriate formulas we identify C ?ω (?X, V (λ)) ? = C ?ω (K )M . The image of Fλ under this identi?cation is independent of λ and will be denoted by F . By [50, Thm. 10.1.5], there are nonvanishing polynomial maps b : a? C→C ? K and D : aC → U (g) , such that (17)

λ?4ρ ? ? b(λ)J (D (λ)) . λ = Jλ?4ρ ? π

SELBERG ZETA FUNCTION

651

The family of di?erential operators π λ?4ρ (D (λ)) : F → F is a holomorphic family of continuous maps. Thus a proof of the lemma for Re(λ) < ? also ?λ in its implies a proof for Re(λ) < ? + 4ρ. It is therefore su?cient to study J domain of convergence Re(λ) < 0. ?λ into a diagonal part J ?1 and an o?We are going to decompose resF ? J λ ?2 . Let F1 be closed such that F ? int(F1 ) ? F1 ? int(W ). diagonal part J λ ? such that F wκ(N ? \ V )M ? F1 M . Let Then there is a compact set V ? N ? \ U )M ? W M , and choose a cut-o? function U ? int(V ) be such that F wκ(N ∞ ? χ ∈ C (N ) such that χ|U = 1 and χ|N ? \V = 0. If f belongs to the dense subspace C ω (?X ) ? F , then we set

1 Jλ (f )(k) 2 Jλ (f )(k)

:= :=

? N ? N

fλ (kwn ? )(1 ? χ(? n))dn ?, fλ (kwn ? )χ(? n)dn ?, k ∈ FM ,

where we employ the notation fλ introduced in (12). We have to show that these operators extend to continuous operators from F to C ω (F M )M , and that this extension depends meromorphically on λ. For any multi-index r ,

1 Jλ (f ) r,F def

=

k ∈F M

sup |

? N

fλ (Xr kwn ? )(1 ? χ(? n))dn ?|

? N

≤ ≤

k ∈W M

sup |f (Xr k)|

r,W

a(? n)Reλ?ρ dn ?

C f

,

where C does not depend on r and f . Recall the de?nition of the R-norm (11). 1 (f ) If R > 0 is su?ciently small (depending on f ), then Jλ R,F ≤ C f R,W . Thus for ?xed λ, Re(λ) < 0, there is a continuous extension 1 : F → C ω (F M )M . An argument similar to the one given in the proof of Jλ 1 depends holomorphically on λ. Lemma 2.16 shows that Jλ ? be the inverse of the analytic di?eomorphism Let ψ : K/M \ M → N ω n ? → wκ(? n)M . If f ∈ C (?X ) and x ∈ F M , then we can write

2 Jλ (f )(x)

= = =

? N ? N K

fλ (xwn ? )χ(? n)dn ? f (xwκ(? n))a(? n)λ?ρ χ(? n)dn ? f (xk)a(ψ (k))λ+ρ χ(ψ (k))dk .

Let G be de?ned in the same way as F (see (16)), but with W replaced by ? \ int(U )) ? supp(1 ? χ ? ψ ). There is a continuous injection of F wκ(N into the space of analytic functions from F M to G given by f → af , af (x) := f (x) ∈ G , x ∈ F M . The multiplication by χ ? ψ is a continuous operator

652

ULRICH BUNKE AND MARTIN OLBRICH

mχ : G → C ?ω (supp(χ ? ψ )). Since the restriction of a(ψ (.))λ+ρ to supp(χ ? ψ ) is analytic, it de?nes a continuous functional on C ?ω (supp(χ ? ψ )). Now we 2 to F by obtain a continuous extension of Jλ (18)

2 Jλ (f )(x) := a(ψ (.))λ+ρ , mχ (af (x)) . 2 ω It is now easy to verify that the map a? C ? λ → Jλ ∈ Hom(F , C (F )) is con2 tinuous and that for ?xed f and x the function λ → Jλ (f )(x) is holomorphic. 2 is holomorphic. Thus by Lemma 2.1 the family of maps Jλ

?λ , Sλ to real analytic sections of Lemma 2.20. (i) The restrictions of S VB (λ) form meromorphic families of continuous maps ?λ , Sλ ∈ Hom(C ω (B, VB (λ)), C ω (B, VB (?λ))) . S ?λ and Sλ in the region Re(λ) < 0 are at most (ii) The singularities of S ?nite-dimensional. (iii) The adjoint ?′ : C ?ω (B, VB (λ)) → C ?ω (B, VB (?λ)) S λ of ?λ to C ?ω (B, VB (λ)). The same holds provides a continuous extension of S true for Sλ . (iv) The following functional equation holds on C ?ω (B, VB (?λ)) : Sλ ? S?λ = id. Proof. (i) and (ii): Let p : ? → B be the projection. We choose compact subsets F ? int(W ) ? W ? ? such that p(F ) = B and de?ne Fλ as above. It follows from Lemma 2.18 (v), that extλ : C ω (B, VB (λ)) → C ?ω (?X, V (λ)) is a meromorphic family of continuous maps with at most ?nite-dimensional singularities. Since resλ ? extλ = id we have a holomorphic family of continuous maps resW ? extλ : C ω (B, VB (λ)) → C ω (W, V (λ)). Thus extλ : C ω (B, VB (λ)) → Fλ is meromorphic with at most ?nite-dimensional singularities. We can identify C ω (B, VB (λ)) with a closed subspace of C ω (F, V (λ)). With this identi?cation, resF = resλ . Assertions (i) and (ii) now follow from Lemma 2.19. ?′ is the continuous extension of S ?λ de?ned (iii) and (iv): The adjoint S λ on distribution sections. The assertions now follow from Lemma 2.18 (iv) and (iii). 2.6. Extension of hyperfunctions and the embedding trick. In this subsection we want to construct the meromorphic continuation of the extension of hyperfunction sections of VB (λ). ?λ : C ω (B, VB (λ)) → C ω (B, VB (?ω )) S

SELBERG ZETA FUNCTION

653

Lemma 2.21.

If δΓ < 0, then

extλ ∈ Hom(C ?ω (B, VB (λ)), C ?ω (?X, V (λ))) (initially de?ned for Re(λ) > δΓ ) admits a meromorphic continuation to all of C with at most ?nite-dimensional singularities. Moreover, extλ has values in Γ C ?ω (?X, V (λ)). Proof. If ?Re(λ) > δΓ , then using Lemma 2.20 (i), (iii) we de?ne extλ := J?λ ? ext?λ ? Sλ . We have by Lemma 2.20 (iv): resλ ? extλ = resλ ? J?λ ? ext?λ ? Sλ = S?λ ? Sλ = id . Since for 0 < Re(λ) < ?δΓ and Im(λ) = 0 the restriction map resλ is injective by Lemma 2.13 and resλ ? extλ = resλ ? extλ = id we conclude that extλ = extλ . Thus extλ is a meromorphic continuation of extλ (de?ned on hyperfunction sections). Note that the family extλ de?ned on hyperfunction sections as constructed above is just the continuous extension of the previously obtained family extλ de?ned on distribution sections. Since extλ considered as an operator on distribution sections has at most ?nite-dimensional singularities by Lemma 2.18 (v), its continuous extension to hyperfunction sections has the same singularities which are, in particular, ?nite-dimensional. We now show how to drop the assumption δΓ < 0 using the embedding trick. Proposition 2.22. Lemma 2.21 holds true without the assumption δΓ < 0. Proof. We realize SO(1, n)0 as the group of automorphisms of Rn+1 ?xing the quadratic form given by diag(?1, 1, . . . , 1). We view SO(1, n)0 as the

n×

subgroup of SO(1, n + 1)0 which ?xes the last element of the standard base of Rn+2 . This embedding is compatible with the standard Cartan involution g → t g ?1 . We set Gn := SO(1, n)0 . Then we have a sequence Gn ?→ Gn+1 , n ≥ 2, of embeddings inducing embeddings of the corresponding Iwasawa constituents ? = K n ?→ K n+1 , N n ?→ N n+1 , M n ?→ M n+1 , and An ?→ An+1 . In particular we identify the Lie algebras an of An in a compatible way with R. The embedding Gn ?→ Gn+1 induces a totally geodesic embedding X n ?→ X n+1 and an embedding of the geodesic boundaries ?X n ?→ ?X n+1 . All these embeddings are equivariant with respect to the action of Gn . If we view the discrete subgroup Γ ? Gn as a subgroup of Gn+1 , then it is still convex cocompact. By ?n , ?n+1 , Λn , Λn+1 we denote the corresponding domains of discontinuity and limit sets. Under the embedding ?X n ?→ ?X n+1 the limit set Λn is identi?ed with Λn+1 . Moreover, we have an embedding ?n ?→ ?n+1 inducing the embedding of compact quotients B n ?→ B n+1 .

654

ULRICH BUNKE AND MARTIN OLBRICH

n . We have the The exponent of Γ now depends on n and is denoted by δΓ n + m n+1 n ? 1 . Thus for m su?ciently large, δ < 0. relation δΓ = δΓ Γ 2 n Let extλ denote the extension map associated to Γ ? Gn . The aim of the +1 leads following discussion is to show how the meromorphic continuation extn λ to the continuation of extn . λ Let P n := M n An N n , V (λ)n := Gn ×P n Vλ , and VB n (λ) = Γ\V (λ)n | ?n . The representation Vλ of P n+1 restricts to the representation Vλ? 1 of P n . 2 This induces isomorphisms of bundles

1 n +1 ? V (λ)n |?X n = V (λ ? 2 ) , Let and

1 VB n+1 (λ)|B n ? = VB n (λ ? ) . 2

1 i? : C ω (B n+1 , VB n+1 (λ)) → C ω (B n , VB n (λ ? )) 2

1 j ? : C ω (?X n+1 , V (λ)n+1 ) → C ω (?X n , V (λ ? )n ) 2 ? denote the maps given by restriction of sections. Note that j is Gn -equivariant. The adjoint maps de?ne the push forward of hyperfunction sections i? : C ?ω (B n , VB n (λ)) → 1 C ?ω (B n+1 , VB n+1 (λ ? )) , 2 1 C ?ω (?X n+1 , V (λ ? )n+1 ) . 2

j? : C ?ω (?X n , V (λ)n ) → Bn

If φ ∈ C ?ω (B n , VB n (λ)), then the push forward i? φ has support in +1 +1 ? B n+1 . Since resn ? extn = id, λ? 1 λ? 1

2 2

(19)

+1 supp(extn λ? 1 2

? i? )(φ) ? Λn+1 ∪ ?n = ?X n .

2

n+1 We are now going to continue extn λ using i? , extλ? 1 and a left inverse of j? .

As on previous occasions we trivialize V (λ)n+1 by powers of a K n+1 -invariant volume form. We thus identify C ?ω (?X n+1 , V (λ)n+1 ) with C ?ω (?X n+1 ) for all λ ∈ C. Let U ? ?X n+1 be a closed tubular neighbourhood of ?X and ? = ?x an analytic di?eomorphism T : [?1, 1] × ?X n → U . Then we can de?ne a continuous extension t : C ω (?X n ) → C ω (U ) by (T ? ? t)f (r, x) := f (x), r ∈ [?1, 1], x ∈ ?X n . Let t′ : C ?ω (U ) → C ?ω (?X n ) be the adjoint of t. Then t′ ? j? = id. Because of (19) we can de?ne

+1 extλ φ := (t′ ? extn ? i? )(φ) . λ? 1

2

n

Then

extλ ∈ Hom(C ?ω (B n , VB n (λ)), C ?ω (?X n , V (λ)n )) is a meromorphic family of continuous maps with at most ?nite-dimensional singularities.

n

SELBERG ZETA FUNCTION n

655

In order to prove that extλ provides the desired meromorphic continuation n it remains to show that it coincides with extn λ in the region Re(λ) > δΓ . If n+1 n+1 1 n n Re(λ) > δΓ , then Re(λ) ? 2 > δΓ , and the push-down maps π?,?λ , π?,?λ+ 1 are de?ned. It is easy to see from the de?nition of the push-down that

n+1 n ? i? ? π? = π? , ?λ ? j . ,?λ+ 1

2 2

Taking adjoints we obtain

n

+1 extn λ? 1 2

2

? i? = j? ? extn λ . Therefore,

n

+1 n extλ = t′ ? extn ? i? = t′ ? j? ? extn λ = extλ . λ? 1

It follows by meromorphy that im(extλ ) consists of Γ-invariant sections for all λ ∈ C. This ?nishes the proof of the proposition. Lemma 2.23. The identities 2.18 (vi), are valid on hyperfunction sections. Proof. This follows by continuity. 3. Green’s formula and applications 3.1. Asymptotics of Poisson transforms. In this subsection we recall facts about the asymptotic behaviour of the Poisson transform Pλ φ ∈ C ∞ (X ) near ?X , where φ ∈ C ?ω (?X, V (λ)). Lemma 3.1. Let ?X = U ∪ Q, where U is open and Q = ?X \ U . Then ?λ : C ?ω (Q, V (λ)) → for any closed subset F ? U the composition resF ? J C ω (F, V (?λ)) extends to a holomorphic family of continuous maps. Proof. Choose a closed set W with nonempty interior such that F ? W ? U . Then C ?ω (Q, V (λ)) ? Fλ (see (16)), and we see by Lemma 2.19 ?λ : C ?ω (Q, V (λ)) → C ω (F, V (?λ)) is a meromorphic family of that resF ? J continuous maps. We argue that it is in fact holomorphic. In the course of the ?λ = J ?1 + J ?2 , proof of Lemma 2.19 we have constructed a decomposition resF ? J λ λ 2 ? admits a holomorphic continuation to all of C by valid for Re(λ) < 0, where J λ ?1 ?ω ? ?2 (18). Since J λ |C (Q,V (λ)) ≡ 0, we conclude that resF ? Jλ = Jλ is holomorphic. Lemma 3.2. (i) Let f ∈ C ∞ (?X, V (λ)). If Re(λ) > 0, then there exists ε > 0 such that for a → ∞, we have (Pλ f )(ka) = aλ?ρ c(λ)f (k) + O(aλ?ρ?ε ) uniformly in k ∈ K . If Re(λ) = 0, λ = 0, then uniformly in k ∈ K .

?λ f (k) + O(a?ρ?ε ) (Pλ f )(ka) = aλ?ρ c(λ)f (k) + a?λ?ρ J

656

ULRICH BUNKE AND MARTIN OLBRICH

(ii) Let ?X = U ∪ Q, where U is open and Q := ?X \ U . Let f ∈ C ?ω (Q, V (λ)). Then there exist smooth functions ψn on U such that (20) ?λ f )(k) + (Pλ f )(ka) = a?(λ+ρ) (J

n≥1

a?(λ+ρ)?n ψn (k) ,

k∈U .

The series converges uniformly for a ? 0 and kM in compact subsets of U . In particular, there exists ε > 0 such that for a → ∞, ?λ f )(k) + O(a?λ?ρ?ε ) (Pλ f )(ka) = a?λ?ρ (J uniformly as kM varies in compact subsets of U . (iii) Let U, Q be as in (ii) and f ∈ C ?ω (?X, V (λ)) such that resU f ∈ C ∞ (U, V (λ)). If Re(λ) > 0, then there exists ε > 0 such that for a → ∞, (Pλ f )(ka) = aλ?ρ c(λ)f (k) + O(aλ?ρ?ε ) uniformly as kM varies in compact subsets of U . If Re(λ) = 0, λ = 0, then there exists ε > 0 such that for a → ∞, ?λ f )(k) + O(a?ρ?ε ) (Pλ f )(ka) = aλ?ρ c(λ)f (k) + a?λ?ρ (J uniformly as kM varies in compact subsets of U . The asymptotic expansions can be di?erentiated with respect to a. Proof. Assertion (i) is a simple consequence of the general results concerning the asymptotics of eigenfunctions on symmetric spaces [2, Thms. 3.5 and 3.6], combined with the limit formulas for the Poisson transform (see [45, Thm. 5.1.4], [49, Thm. 5.3.4]):

a→∞ a→∞

lim aρ?λ (Pλ f )(ka) =

c(λ)f (k), ?λ f )(k), (J

Re(λ) > 0 , Re(λ) < 0 .

lim aρ+λ (Pλ f )(ka) =

For f ∈ C (?X, V (λ)), supp(f ) ? Q, assertion (ii) follows from Theorem 4.8 in [3]. However, the proof given in that paper works as well for f ∈ C ?ω (Q, V (λ)). (iii) is a consequence of (i) and (ii). Indeed, let W, W1 ? U be compact ∞ (U ) be such that χ subsets such that W ? int(W1 ). Let χ ∈ Cc |W1 ≡ 1. Then we can write f = χf + (1 ? χ)f , where χf is smooth and supp(1 ? χ)f ? ?X \ int(W1 ). We now apply (i) to χf and (ii) to (1 ? χ)f for kM ∈ W . 3.2. An orthogonality result. The following facts were shown in [8, §6]. Lemma 3.3. Let F ? X ∪ ? be a fundamental domain for Γ. There exists a cut-o? function χ ∈ C ∞ (X ) such that :

SELBERG ZETA FUNCTION

657

g ∈L

(i) There is a ?nite subset L ? Γ such that supp(χ) ? (ii)

g ∈Γ g ?χ

gF ;

= 1;

(iii) For any i ∈ N and compact W ? ?, supk∈W M, a∈A+ a |?i χ(ka)| < ∞; (iv) χ extends continuously to ?X and the restriction χ∞ of this extension to ? is a smooth function with compact support. Let φ ∈ Γ C ?ω (Λ, V (λ)). Then by Lemma 3.1 the composition res?λ ? ?λ (φ) ∈ Γ C ω (?, V (?λ)) = C ω (B, VB (?λ)) is well-de?ned for all λ ∈ C. J Proposition 3.4. Γ C ?ω (?X, V (λ)), then If Re(λ) ≥ 0, φ ∈

Γ C ?ω (Λ, V

(λ)), and f ∈

?λ (φ), resλ (f ) = 0 . res?λ ? J Proof. At ?rst we need the following: Lemma 3.5. The space

Γ ?ω C? (?X, V (λ)) := {f ∈ Γ C ?ω (?X, V (λ)) | f|? ∈ C ∞ (?, V (λ))}

is dense in Γ C ?ω (?X, V (λ)). Proof. By Proposition 2.22 the family ext? has an at most ?nite-dimensional singularity at ? = λ. Thus there is a ?nite-dimensional subspace W ? C ω (B, VB (?λ)) such that (extλ )|W ⊥ : W ⊥ → Γ C ?ω (?X, V (λ)) is a well-de?ned continuous map, where W ⊥ := {φ ∈ C ?ω (B, VB (λ)) | φ, W = {0}}. Since ? ? C ∞ (B, VB (λ)) ? C ?ω (B, VB (λ)) is dense we can choose a complement W ∞ ? ω ⊥ ? C (B, VB (λ)) such that C (B, VB (λ)) = W ⊕ W . Let f ∈ Γ C ?ω (?X, V (λ)). Then we can write resλ f = g = g⊥ ⊕ g ?, ⊥ ⊥ ⊥ ? g ∈ W , g ? ∈ W . Now resλ (f ? extλ (g )) = g ?. It follows that f ? ?ω extλ (g⊥ ) ∈ Γ C? (?X, V (λ)). Let now gi ∈ C ∞ (B, VB (λ)) be a sequence such ⊥ +g ⊥ are that limi→∞ gi = g. Then we can decompose gi = gi ?i . The sections gi ? ω Γ ⊥ ? are so. It follows that extλ (g ) ∈ C (?X, V (λ)). smooth since gi and g ?i ∈ W i ? ⊥ ). The asserBy continuity of (extλ )|W ⊥ we have extλ (g⊥ ) = limi→∞ extλ (gi ⊥ ). tion of the lemma now follows from f = f ? extλ (g⊥ ) + limi→∞ extλ (gi

?ω (?X, V (λ)). By Lemma 3.5 it su?ces to prove the proposition for f ∈ Γ C? By Lemma 2.13 we can assume that λ ∈ [0, ∞) ∪ ?R. Let A := ?X ? ρ2 + λ2 , P := Pλ be the Poisson transform, and let χ be the cut-o? function given by Lemma 3.3. By BR ? X we denote the metric ball of radius R centered at the origin represented by K .

658

ULRICH BUNKE AND MARTIN OLBRICH

Note that AP f = 0 and AP φ = 0 (see subsection 2.4). Integration by parts gives (21) 0 = = = χAP φ, P f AχP φ, P f

L2 (BR ) L2 (BR )

? χP φ, AP f

L2 (BR ) L2 (BR )

? [A, χ]P φ, P f

? ?n χP φ, P f

? χP φ, AP f ,

L2 (?BR ) L2 (BR )

+ χP φ, ?n P f

? [A, χ]P φ, P f

L2 (BR )

L2 (?BR )

where n is the exterior unit normal vector ?eld at ?BR . For the following discussion we distinguish between the three cases: (i) λ ∈ (0, ∞), (ii) Re(λ) = 0, λ = 0, and (iii) λ = 0. We ?rst consider the case λ > 0. Lemma 3.3 (iii) implies that |[A, χ]P φP f | is integrable over all of X , and by Lemma 3.3 (ii), Lemma 3.2 (ii), (iii) and the Γ-invariance of f, φ, [A, χ]P φ, P f

L2 (X )

=

γ ∈Γ

γ ? ([A, χ])P φ, P f

L2 (Y )

=0.

Taking the limit R → ∞ in (21), and using Lemma 3.2 (i), (ii) and Lemma 3.3, we obtain 0 = (λ + ρ)

K

? χ∞ (k)(J λ φ)(k )c(λ)f (k )dk

K

+(λ ? ρ) = = 2λc(λ)

K

? χ∞ (k)(J λ φ)(k )c(λ)f (k )dk

? χ∞ (k)(J λ φ)(k )f (k )dk

?λ (φ), resλ (f ) . 2λc(λ) res?λ ? J

This is the assertion of the proposition for λ > 0 since c(λ) = 0 (see [19, Ch. IV, Thm. 6.14]). Now we discuss the case Re(λ) = 0 and λ = 0. By Lemma 3.2 (i), the function P (f )(ka) now has two leading terms. Instead of taking the limit r R → ∞ in (21) we apply limr→∞ 1 r 0 dR. Again we have 1 r →∞ r lim

r

[A, χ]P φ, P f

0

L2 (BR ) dR

=0.

?λ f (k) of P (f )(ka) does not contribute to the limit The leading term a?λ?ρ J since 1 r ?2λ?2ρ ?λ φ, J ? R χ(R)J ?2λ lim λ f L2 (?X ) dR = 0 . r →∞ r 0

SELBERG ZETA FUNCTION

659

The contribution of the term aλ?ρ c(λ)f (k) leads to ?λ (φ), resλ (f ) 0 = 2λc(λ) res?λ ? J as in the case λ > 0. The proposition again follows since c(λ) = 0. Now we consider the last case λ = 0. The function c(λ) has a ?rst order pole at λ = 0 with residue c1 = 0 (see [19, Ch. IV, Thm. 6.14]). We redo the computation (21) using the Poisson transform P = P? at ? in a neighbourhood of 0. Of course in general P? (f ), P? (φ) are not Γ-invariant except for ? = 0. Nevertheless, lim [A, χ]P1/R φ, P1/R f L2 (BR ) = 0

R→∞

by the theorem of Lebesgue about dominated convergence. Moreover, ? lim ( ?n χP1/R φ, P1/R f

R→∞ L2 (?BR )

? χP1/R φ, ?n P1/R f

L2 (?BR ) )

e2ρR vol(?BR )

=

R→∞

lim ((1/R + ρ)

K K

? χ∞ (k)(J 1/R φ)(k )c(1/R )f (k )dk

+(1/R ? ρ) = = 2

? χ∞ (k)(J 1/R φ)(k )c(1/R)f (k )dk )

R→∞

c(1/R) f (k)dk R K ?0 (φ), res0 (f ) . 2c1 res0 ? J ? χ∞ (k)(J 0 φ)(k ) lim

It follows from (21) that the latter pairing vanishes. This ?nishes the proof of the proposition in the last case λ = 0. 3.3. Miscellaneous results. The following lemma is very similar to the general result [3, Thm. 4.1]. Lemma 3.6. Let U ? ?X be a nonempty open subset. Assume that λ ∈ C satis?es λ ∈ ?ρ ? N0 ∪ ?N. If φ ∈ C ?ω (?X, V (λ)) satis?es resU (φ) = 0 and ?λ (φ) = 0, then φ = 0 (note that (J ? ? resU ? J λ φ)|U is de?ned even if Jλ has a pole ). ?λ (φ), Proof. We reduce the proof to the case Re(λ) ≥ 0 by replacing φ by J ?λ and J ??λ if Re(λ) < 0. We can do so because λ ∈ ?ρ ? N0 ∪ ?N and thus J are regular and bijective (see Lemma 4.13 below). We now assume that Re(λ) ≥ 0. Then the Poisson transform Pλ is a bijection between C ?ω (?X, V (λ)) and ker(D ), where D := ?X ? ρ2 + λ2 . Since D is elliptic with real analytic coe?cients Pλ φ is a real analytic function. We argue by contradiction and assume that φ ≡ 0. Without loss of generality we can assume that M ∈ U . Since Pλ φ is real analytic and not identically zero the expansion (20) has nontrivial terms. Let m be the smallest integer

660

ULRICH BUNKE AND MARTIN OLBRICH

?λ φ. The contradiction will be obsuch that ψm ≡ 0 near M , where ψ0 := J tained by showing that m = 0. With respect to the coordinates k, a the operator D has the form D = D0 + a?α1 R(a, k), where D0 is a constant coe?cient operator on A and R is a di?erential operator with coe?cients which remain bounded if a → ∞ (see [19, ? -radial Ch. IV, §5, (8)]). Moreover, it is known that D0 coincides with the N part of D . ? -invariant function f ∈ C ∞ (X ) de?ned by f (? We consider the N na) ? ( λ + ρ + m ) := a . Since D annihilates the asymptotic expansion (20) we have Df = D0 f = 0. On the other hand, f satis?es (?X ? ρ2 + (λ + m)2 )f = 0. Hence (λ + m)2 = λ2 . Since Re(λ) ≥ 0 we conclude that m = 0. The following corollary is an immediate consequence of Lemma 3.6 and Lemma 3.1. Corollary 3.7. is injective. Lemma 3.8. If λ > 0, then the order of a pole of ext? at λ is at most 1. Proof. Let f? ∈ C ω (B, VB (?)), ? ∈ C, be a holomorphic family such that ext? (f? ) has a pole of order l ≥ 1 at ? = λ, λ > 0. We assume that l ≥ 2 and argue by contradiction. Let 0 = φ ∈ Γ C ?ω (?X, V (λ)) be the leading singular part of ext? (f? ) at ? = λ. Because of res? ? ext? = id we have resλ φ = 0 and hence φ ∈ Γ C ?ω (Λ, V (λ)). By Lemma 3.2 for any compact subset F ? ? there exist constants C1 , C2 , C3 such that for a ? 0, k ∈ F M , ? near λ, 0 < ? < λ, |Pλ φ(ka)| ≤ C1 a?λ?ρ and (22) |(? ? λ)l P? ext? (f? )(ka)| ≤ C3 (1 + log a)?l aλ?ρ . ≤ C2 ((λ ? ?)l a??ρ + a?ρ ) If λ ∈ ?ρ ? N0 ∪ ?N, then ?λ : C ?ω (Λ, V (λ)) → C ω (?, V (?λ)) res? ? J

In particular, Pλ φ ∈ L2 (Y ). Since l ≥ 2 we obtain by Lebesgue’s theorem of dominated convergence ||Pλ φ||2 = =

?→λ ?<λ

lim (? ? λ)l P? ext? (f? ), Pλ φ

L2 (Y ) L2 (Y )

?→λ ?<λ

lim (? ? λ)l P? ext? (f? ), Pλ φ

.

On the other hand the estimates (22) allow partial integration, and we obtain for ? < λ,

SELBERG ZETA FUNCTION

661

P? ext? (f? ), Pλ φ

= = =

1 (?Y ? ρ2 + λ2 )P? ext? (f? ), Pλ φ λ2 ? ?2 1 P? ext? (f? ), (?Y ? ρ2 + λ2 )Pλ φ λ2 ? ?2 0.

Hence ||Pλ φ||L2 (Y ) = 0, which is a contradiction. We conclude that l = 1. For λ ∈ ?R there is a conjugate-linear pairing Vλ ? Vλ → V?ρ and hence a natural L2 -scalar product C ∞ (B, VB (λ)) ? C ∞ (B, VB (λ)) → C. Let L2 (B, VB (λ)) be the associated Hilbert space. Using Lemma 2.18 (iv), we see ? with respect to this Hilbert space structure is just S that the adjoint Sλ ?λ . Lemma 3.9. If Re(λ) = 0, then Sλ is regular and unitary. (iii), Proof. Assume that Sλ is regular. If f ∈ C ∞ (B, VB (λ)), then by Lemma 2.18 Sλ f

2 L2 (B,VB (λ))

= S?λ ? Sλ f, f

L2 (B,VB (λ))

= f

2 L2 (B,VB (λ))

.

Thus Sλ is unitary. Meromorphy of Sλ and Sλ = 1 for all λ ∈ ?R with Sλ regular imply regularity of Sλ for all λ ∈ ?R. Lemma 3.10. If Re(λ) = 0, λ = 0, then extλ : C ?ω (B, VB (λ)) → Γ C ?ω (?X, V (λ)) is regular. Proof. By Lemma 3.9 the scattering matrix Sλ is regular for Re(λ) = 0. ?λ . Recall Since c(?λ) is regular for Re(λ) = 0, λ = 0, the same holds true for S ?λ = res?λ ? J ? that S λ ? extλ . Corollary 3.7 and the fact that the leading singular part of extλ maps to hyperfunctions supported on the limit set Λ show that a ?λ . Thus extλ is regular pole of extλ would necessarily imply a singularity of S for Re(λ) = 0, λ = 0. Lemma 3.11. (i) At λ = 0 the family extλ has at most a ?rst order pole. (ii) ext0 is regular if and only if S0 = id. (iii) The operator S0 ? id is ?nite-dimensional. Proof. Since S0 is regular and unitary by Lemma 3.9 and c(λ) has a ?rst ?λ has a ?rst order order pole at λ = 0 the unnormalized scattering matrix S pole at λ = 0, too. The principal series representation of G on C ∞ (?X, V (0)) is irreducible ([25, Cor. 14.30] or [20, Ch.VI, Thm. 3.6]). Thus J0 = id by our choice of normalization.

662

ULRICH BUNKE AND MARTIN OLBRICH

k k 0 ?λ = (resλ=0 c(?λ)) λ?1 + J 0 and extλ = We expand J k<0 λ ext + extλ , λ 0 and ext0 depend holomorphically on λ near λ = 0. Since for k < 0 where Jλ λ we have supp(extk (f )) ? Λ for all f ∈ C ?ω (B, VB (0)) Corollary 3.7 implies 0 ? extk = 0 if and only if extk = 0. Since S ?λ has a ?rst order pole that res? ? J0 k it follows that ext = 0 for k ≤ ?2, hence (i). We now consider the residue of ?λ . S

resλ=0 c(?λ)S0

= = =

?λ resλ=0 S ?λ ? extλ resλ=0 res?λ ? J

0 resλ=0 c(?λ)id + res0 ? J0 ? ext?1 .

0 ? ext?1 . The right-hand We conclude that resλ=0 c(?λ)(S0 ? id) = res0 ? J0 side of this equation is a ?nite-dimensional operator, and (iii) follows. Again by Corollary 3.7 we conclude that ext?1 = 0 if and only if S0 = id. This is assertion (ii).

Γ C ?ω (?X, V

Lemma 3.12. If Re(λ) ≥ 0, then the residue of ext? : C ?ω (B, VB (?)) → (?)) at ? = λ spans Γ C ?ω (Λ, V (λ)). In particular, ext? is regular at λ if and only if Γ C ?ω (Λ, V (λ)) = 0. Proof. By Lemmas 3.8, 3.11 and 3.10 for Re(?) ≥ 0 the family ext? has poles of at most ?rst order. Thus res?=λ ext? has values in Γ C ?ω (Λ, V (λ)). Let d := dim im res?=λ ext? ≤ dim Γ C ?ω (Λ, V (λ)). The meromorphic identity res? ? ext? = id now implies that d ≥ dim coker resλ . By Proposition 3.4 and Corollary 3.7, dim Γ C ?ω (Λ, V (λ)) = dim(im resλ )⊥ ≥ dim Γ C ?ω (Λ, V (λ)) . It follows that d = dim Γ C ?ω (Λ, V (λ)). This proves the lemma. ≥ d ≥ dim coker resλ

Corollary 3.13. The set F := {λ ∈ C|Re(λ) ≥ 0, Γ C ?ω (Λ, V (λ)) = 0} is ?nite and contained in [0, δΓ ]. Moreover F is the set of singularities of extλ for Re(λ) ≥ 0. If λ ∈ F then ext? has a ?rst order pole at ? = λ. Proof. We combine Lemmas 2.13, 3.10, and 3.12. 4. Cohomology 4.1. Hyperfunctions with parameters. In this subsection we compare di?erent de?nitions of holomorphic families of hyperfunctions. Let M be a real analytic manifold and K ? M be a compact subset. Then the space C ?ω (K ) of hyperfunctions on M with support in K has a natural Fr? echet topology as the strong dual of the space of germs of

SELBERG ZETA FUNCTION

663

real analytic functions at K . Thus we can consider holomorphic families of hyperfunctions in C ?ω (K ) as in subsection 2.2. If U ? C, then by OC ?ω (K )(U ) we denote the space of holomorphic functions from U with values in C ?ω (K ). Hyperfunctions on M form a ?abby sheaf BM . Thus we can consider the space of hyperfunctions BM (V ) on an open subset V ? M . This space does not carry a natural topology. In order to de?ne holomorphic families of hyperfunctions on V we have to follow a di?erent approach. Let Mc be a complex neighbourhood of M and OMc be the sheaf of holomorphic functions n (O ? on Mc . Then by de?nition BM = HM Mc ), where n = dim(M ) and HM (OMc ) denotes the relative cohomology sheaf [27, p. 192?]. It is a theorem (see [27, p. 219]) that {φ ∈ BM (M ) | supp(φ) ? K } = C ?ω (K ) in a natural manner. If V ? M is open, then a hyperfunction f ∈ BM (V ) can be represented (in a nonunique way) as a locally ?nite sum of hyperfunctions with compact support contained in V . Using the approach via relative cohomology we can de?ne the sheaf of holomorphic families of hyperfunctions as follows. Consider the embedding n C × M ?→ C × Mc . Then we de?ne the sheaf OBM := HC ×M (OC×Mc ) on C × M . If U ? C, V ? M are open, then OBM (V )(U ) := OBM (U × V ) is by de?nition the space of holomorphic families of hyperfunctions on V parametrized by U . An important consequence of this de?nition is the following property. Let T ? C be Stein. Then the prescription M ? V → OBM (V )(T ) de?nes a ?abby sheaf on M (see [33, §5]). ? be the Cauchy-Riemann operator acting on the ?rst variable of Let ? C × M . Then we have an alternative description of the sheaf OBM as the ?; i.e., OBM (W ) = {φ ∈ BC×M (W ) | ?φ ? = 0} (see [27, solution sheaf of ? p. 308]). If K ? M is compact, then we can consider OBM (K )(U ) := {φ ∈ OBM (M )(U ) | supp(φ) ? U × K } . Lemma 4.1. There is a natural isomorphism OC ?ω (K )(U ) ? = OBM (K )(U ). Proof. In this proof we employ the description of OBM as the solution ?-equation. First we de?ne a map Φ : OC ?ω (K )(U ) → sheaf of the partial ? OBM (K )(U ). Let {Uα } be a locally ?nite covering of U and let {χα } be a subordinated partition of unity. If f ∈ OC ?ω (K )(U ), then χα f can be ? × K with support in supp(χα ) × K considered as an analytic functional on U ω ? as follows: For φ ∈ C (U × K ) we set χα f, φ := where φλ := φ(λ, .) ∈ C ω (K ).

U

χα (λ) fλ , φλ dλ ,

664

ULRICH BUNKE AND MARTIN OLBRICH

Then we de?ne Φ(f ) ∈ BC×M (U × K ) to be the hyperfunction represented by the locally ?nite sum α χα f ∈ BC×M (U × M ) of hyperfunctions with compact support. Since the functions χα form a partition of unity on U and ? fλ depends holomorphically on λ it is easy to see that ? α χα f = 0. Thus indeed Φ(f ) ∈ OBM (U × K ). We now construct the inverse Ψ : OBM (K )(U ) → OC ?ω (K )(U ). Let ? = 0. ThereF ∈ OBM (K )(U ). Since F ∈ BC×M (U × M ) it satis?es ?F fore the specialization F (λ, .) ∈ BM (M ) is de?ned for all λ ∈ U . Clearly supp(F (λ, .)) ? K and we de?ne Ψ(f )λ := F (λ, .). We must show that Ψ is well-de?ned, i.e. that U ? λ → Ψ(F )λ ∈ C ?ω (K ) is a holomorphic function. From the arguments of the proof of [22, Thm. 4.4], there exists a local elliptic operator J on C × M of possible in?nite order acting on the second ? = 0 such that Ju = F . variable and a function u ∈ C ∞ (U × M ) satisfying ?u By [22, Thm. 4.3], we can view U ? λ → uλ := u(λ, .) as a holomorphic function on U with values in C ∞ (M ). ∞ (M ) be a cut-o? function with χ ≡ 1 in a neighbourhood Let χ ∈ Cc of K . Since J de?nes a continuous operator from C ?ω (supp(χ)) into itself the function λ → J (χuλ ) is holomorphic on U with values in C ?ω (supp(χ)). We can write J (χuλ ) = F (λ, .) + F1 (λ), where supp(F1 )(λ) ? supp(dχ). Since K and supp(dχ) are separated we have a continuous decomposition C ?ω (K ∪ supp(dχ)) = C ?ω (K ) ⊕ C ?ω (supp(dχ)). Thus U ? λ → F (λ, .) is a holomorphic function with values in C ?ω (K ). It remains to show that Φ and Ψ are inverse to each other. Let F ∈ ∞ (U ) is considered as a function on U × M , then since OBM (K )(U ). If χ ∈ Cc ? = 0 the product χF ∈ BC×M (U × M ) is well-de?ned. Since supp(χF ) ? ?F supp(χ) × K is compact, χF ∈ C ?ω (supp(χ) × K ). Using the partition of unity {χα } introduced above we can write F as a locally ?nite sum of analytic functionals F = α χα F . Now we show Φ ? Ψ = id. We have Ψ(F )λ = F (λ, .). Then Φ ? Ψ(F ) is given by α χα F = F . In order to prove that Ψ ? Φ = id, note that Φ(f )(λ, .) = α χα (λ)fλ = fλ . This ?nishes the proof of Lemma 4.1. Fix λ ∈ C. If V ? M is open, then we de?ne

?→

Oλ C ?ω (V ) := lim OBM (V )(U ) ,

where U runs over all open neighbourhoods of λ. Lemma 4.2. The prescription M ? V → Oλ C ?ω (V ) de?nes a ?abby sheaf Oλ BM on M . Proof. There exists a fundamental sequence of Stein neighbourhoods {Tn } of λ. We can write Oλ C ?ω (V ) = lim OBM (V )(Tn ). Recall that M ? V → ? →

n

SELBERG ZETA FUNCTION

665

OBM (V )(Tn ) de?nes a ?abby sheaf on M . A direct limit of ?abby sheaves is again a ?abby sheaf. For later reference we need the following result. Let φ ∈ C ω (M ) be such that 0 is a regular value. Then N := {φ = 0} ? M is a real analytic submanifold of codimension one. Let i : N ?→ M denote the embedding. Then there is a natural morphism of sheaves i? Oλ BN → Oλ BM . Multiplication by φ induces a morphism φ : Oλ BM → Oλ BM . Lemma 4.3. The sequence of sheaves (23) is exact. Proof. Let Mc be a complex neighbourhood of M such that φ extends holomorphically to Mc and 0 remains to be a regular value of this extension. Then Nc := {φ = 0} is a complex neighbourhood of N . We use i to denote the embedding Nc ?→ Mc , too. There is a natural exact sequence of sheaves (24) 0 → OC×Mc → OC×Mc → i? OC×Nc → 0 .

φ

0 → i? Oλ BN → Oλ BM → Oλ BM → 0

φ

Since Nc ? Mc is a locally closed submanifold, i? = i! is an exact functor (see [24, Prop. 2.5.4]). It follows that

p By [27, p. 26], for p = n and q = n ? 1 we have HC ×M (OC×Mc ) = 0 and q HC×N (OC×Nc ) = 0. The long exact sequence in cohomology associated with (24) gives n?1 n n 0 → i? HC ×N (OC×Nc ) → HC×M (OC×Mc ) → HC×M (OC×Mc ) → 0 , φ ? ? ? HC ×M (i? OC×Nc ) = i? HC×(M ∩Nc ) (OC×Nc ) = i? HC×N (OC×Nc ) .

or equivalently 0 → i? OBN → OBM → OBM → 0 . Taking germs at λ ∈ C we obtain (23) and conclude exactness. 4.2. Acyclic resolutions. The main goal of this paper is to compute the cohomology groups H ? (Γ, M), where the coe?cients M are certain Γ-modules (i.e. complex representations of Γ). A Γ-module M is called Γ-acyclic if and only if H p (Γ, M) = 0 for all p ≥ 1. A Γ-acyclic resolution of M is a complex of Γ-modules with H 0 (C . ) = M, H p (C . ) = 0, p ≥ 1, where all C i are Γ-acyclic. Standard homological algebra gives an isomorphism H ? (Γ, M) = H ? (Γ C . ) , C. : 0 → C 0 → C 1 → C 2 → . . . ,

φ

666 where

ULRICH BUNKE AND MARTIN OLBRICH

Γ .

C : 0 → ΓC 0 → ΓC 1 → ΓC 2 → . . . .

Using suitable trivializations of the families V (λ), VB (λ) we can carry over the results of subsection 4.1 to hyperfunction sections of V (λ) and VB (λ). If V ? ?X is open, then let Oλ C ?ω (V ) denote the space of germs at λ of holomorphic families of hyperfunction sections of V (λ) on V . By Lemma 4.2 the prescription ?X ? V → Oλ C ?ω (V ) de?nes a ?abby sheaf. Let res? : Oλ C ?ω (?X ) → Oλ C ?ω (?) be the restriction. Then we de?ne Oλ C ?ω (Λ) := ker res? . Note that Λ and ?X are compact. By Lemma 4.1 we can identify the spaces Oλ C ?ω (Λ) and Oλ C ?ω (?X ) with the spaces of germs at λ of holomorphic functions ? → f? ∈ C ?ω (Λ, V (?)) and ? → f? ∈ C ?ω (?X, V (?)) in the sense of subsection 2.2. Lemma 4.4. (25) is exact. Proof. The restriction res? is surjective since the sheaf V ? ?X → Oλ C ?ω (V ) is ?abby. Exactness at the other places holds by de?nition. Lemma 4.5. The Γ-module Oλ C ?ω (?) is Γ-acyclic. Proof. The group Γ acts freely and properly on ?. The Γ-module Oλ C ?ω (?) is the space of sections of a Γ-equivariant ?abby sheaf on ?. Thus the assertion can be shown by repeating the arguments of the proof of [4, Lemma 2.6]. The proof of the following proposition will occupy the remainder of this subsection. Proposition 4.6. If λ ∈ ?ρ ? N0 , then Oλ C ?ω (?X ) is Γ-acyclic. The following complex of Γ-modules

res

? Oλ C ?ω (?) → 0 0 → Oλ C ?ω (Λ) → Oλ C ?ω (?X ) →

Proof. We will show that H p (Γ, Oλ C ?ω (?X )) = 0 using a suitable acyclic resolution. If U ? C is open, then de?ne ? = 0} , OC ∞ (X )(U ) := {f ∈ C ∞ (U × X ) | ?f ? is the partial Cauchy-Riemann operator acting on the ?rst variable. where ? By [22, Thm. 4.3], OC ∞ (X )(U ) is the space of holomorphic functions from U to the Fr? echet space C ∞ (X ). We de?ne Oλ C ∞ (X ) := lim OC ∞ (X )(U ) ,

?→

where U runs over all neighbourhoods of λ.

SELBERG ZETA FUNCTION

667

We de?ne the operator A : OC ∞ (X )(U ) → OC ∞ (X )(U ) by (Af )? = (?X ? ρ2 + ?2 )f? , where f? = f (?, .). We use the same symbol A in order to denote the induced operator on Oλ C ∞ (X ). Note that the Poisson transform P? comes as a holomorphic family of continuous maps P? : C ?ω (?X, V (?)) → C ∞ (X ). Viewing Oλ C ?ω (?X ) as a space of germs at λ of holomorphic functions with values in a topological vector space we can de?ne a Poisson transform P : Oλ C ?ω (?X ) → Oλ C ∞ (X ) by (P f )? = P? f? . Since (?X ? ρ2 + ?2 )P? f? = 0 we have A ? P = 0. We need to know that P : Oλ C ?ω (?X ) → {f ∈ Oλ C ∞ (X ) | Af = 0} is surjective. Proposition 4.7. Let U ? C be open such that U ∩ ?ρ ? N0 = ?. Let ? → f? , ? ∈ U , be a continuous (smooth, holomorphic ) family of eigenfunctions, i.e., Af = 0. Then there exists a continuous (smooth, holomorphic ) family of hyperfunctions βf , (βf )? ∈ C ?ω (?X, V (?)), such that P (βf ) = f . Proof. In order to prove the proposition one may take any of the existing proofs of the pointwise surjectivity of P? ([18], [23], [46]). One has to control the dependence on ? of the construction of the inverse map β? . We prefer to follow Helgason’s proof [18] because it is the most elementary one. ? M denote the set of equivalence classes of irreducible representations Let K of K with a nontrivial M -?xed vector. Let δp be the class represented by the representation of K on the space of homogeneous harmonic polynomials of ? M = {δp | p ∈ N0 }. Hence there is a decomposition degree p on Rn . Then K

∞

(26)

f? =

p=0

f?,p ,

where f?,p ∈ ker(?X ? ρ2 + ?2 ) transforms according to the K -type δp . For ? ∈ ?ρ ? N0 , any K -?nite eigenfunction is the Poisson transform of a unique K -?nite function on ?X (see e.g. [20, Ch.III, Thm. 6.1]); i.e., f?,p = P? ??,p for a certain ??,p ∈ C ∞ (?X ) .

Here and in the following we identify C ? (?X, V (?)) with C ? (?X ) using a K invariant volume form. Let ψp be the unique M -spherical function on ?X of K -type δp , and set Φ?,p := P? ψp . Then we have ([18, Lemma 4.2]) (27) f?,p (ka) = ??,p (k) · Φ?,p (a) , k ∈ K, a ∈ A .

We want to show that for a continuous family f the series ? → β? f? := p ??,p converges in the space of continuous functions from U to C ?ω (?X ). The resulting limit βf then satis?es P (βf ) = f . In fact, we will show the stronger result that β : {f ∈ C (U, C ∞ (X )) | Af = 0} → C (U, C ?ω (?X )) is continuous.

668

ULRICH BUNKE AND MARTIN OLBRICH

Let ? ∈ C ?ω (?X ) and ? = p ?p be its decomposition with respect to K -types. For 0 < R < 1 we de?ne seminorms ||.||R on C ?ω (?X ) by

∞

||?||2 R :=

p=0

R2p ||?p ||2 L2 (?X ) .

It is not di?cult to see that the seminorms ||.||R , 0 < R < 1, de?ne the topology on C ?ω (?X ) (compare [18, Prop. 5.2]). Thus the topology on C (U, C ?ω (?X )) is given by the seminorms ||?||W,R := sup ||?? ||R ,

?∈W

W ? U compact, 0 < R < 1 .

The function Φ?,p can be represented in terms of the hypergeometric function F = 2 F1 as follows ([18, p. 341]):

1 1 1 1 r2 Φ?,p (a) = cp (?)(1 ? r 2 )ρ? 2 r p F (? + , ?? + , p + ρ + , 2 ), 2 2 2 r ?1

where a ∈ [1, ∞), r =

a?1 a+1

and cp (?) :=

1 Γ(?+ρ+p) Γ(ρ+ 2 ) Γ(?+ρ) Γ(ρ+ 1 +p) . 2

Now ?x R and a compact set W ? U . The following two estimates are crucial: (i) There exist constants C1 and C2 such that for all p ∈ N0 and ? ∈ W

1 1 |Γ(? + ρ)| 1 ≤ C1 (1 + p) 2 ?Re(?) ≤ C2 (1 + p) 2 ?Re(?) . 1 |cp (?)| Γ(ρ + 2 )

In particular, (28) sup

?∈W

1 ≤ C (1 + p)2k |cp (?)|

for some C > 0, k ∈ N0 . (ii) Set y :=

R2 R2 ?1 .

Then there exists P ∈ N0 such that for all p ≥ P , ? ∈ W ,

(29)

1 1 1 1 |F (? + , ?? + , p + ρ + , y )| ≥ . 2 2 2 2

Γ(x) Assertion (i) follows from limx→∞ xα Γ( x+α) = 1 for real x and α ∈ C (see [11, p. 47]). In order to verify (ii) we estimate

?1 (?, p, y )

:= =

1 F (? + , ?? + 2 1 F (?? + , ? + 2

1 ,p + ρ + 2 1 ,p + ρ + 2

1 , y) ? 1 2 1 , y) ? 1 2

SELBERG ZETA FUNCTION

669

? := (W ∪ ?W ) ∩ {Re(?) ≥ 0}. By [11, for ? varying in the compact set W ? and p > sup ? (Re(?) ? ρ), p. 76] we have for ? ∈ W ?∈W |?1 (?, p, y )| ≤ 1 (|y | + 1)Re(?)+1 |?2 ? | · 2 ) Γ(Re(?) + 1 Γ( p + ρ ? Re(?)) 1 2 · 1 |Γ(p + ρ ? ?)| p + ρ + |Γ(? + 2 )| 1 C . p+ρ+ 1 2

1 2

≤

1 If P > sup?∈W ? (Re(?) ? ρ) is large enough, then |?1 (?, p, y )| ≤ 2 for all p ≥ P ? . Assertion (ii) follows. and ? ∈ W Now we are able to estimate ||βf ||W,R for a continuous family of eigenfunctions f . Equations (26) and (27) imply that Q1 de?ned by P ?1

Q1 (f ) := sup (

?∈W p=0

2 R2p ||??,p ||2 L2 (?X ) )

1

is a continuous seminorm on {f ∈ C (U, C ∞ (X )) | Af = 0}. Using (i), (ii) and the fact that δp (CK ) = p(p + n ? 2)id, where CK is the Casimir operator of K , we estimate the remainder

∞ p =P

R2p ||??,p ||2 L2 (?X ) 4 1 1 R2 1 )|2 ||??,p ||2 R2p |F (? + , ?? + , p + ρ + , 2 L2 (?X ) 2 2 2 R ? 1 p =P

∞ p=0 ∞

(29)

≤

(28)

≤

≤

We obtain

1 1 1 R2 1 ·||cp (?)(1 ? R2 )ρ? 2 Rp F (? + , ?? + , p + ρ + , 2 )??,p ||2 L2 (?X ) 2 2 2 R ?1 1+R C′ |(1 + CK )k f? (ka)|2 dk , a= . 1?R K

4C (1 ? R2 )2ρ?1

(1 + p)4k ·

||βf ||W,R ≤ Q1 (f ) + C ′ 2 Q2 (f ) , where Q2 is the continuous seminorm on C (U, C ∞ (X )) given by Q2 (f ) := sup |(1 + CK )k f? (ka)|2 dk

1 2

1

.

?∈W

K

Continuity of β : {f ∈ C (U, C ∞ (X )) | Af = 0} → C (U, C ?ω (?X )) follows. It remains to discuss smooth and holomorphic families of eigenfunctions ? be the Cauchy-Riemann operator acting on the ?-variable. Let f . Let ?

670

ULRICH BUNKE AND MARTIN OLBRICH

?k f is again a smooth family f ∈ C ∞ (U, C ∞ (X )) be such that Af = 0. Then ? of eigenfunctions. Since Φ?,p is holomorphic in ? it follows from (27) that ?l f ) = β (? Hence (30)

∞ ?l p=0 ? ?.,p ∞ p=0

?l ?.,p , ?

for all l ∈ N0 .

converges for all l ∈ N0 , and we conclude that

∞ ∞

?k (βf ) = ? ?k ?

?.,p =

p=0 p=0

?k ?.,p = β (? ?k f ). ?

Thus, if f is holomorphic, then so is βf . If f is smooth, then the right-hand side of (30) is continuous for all k. Hence, by elliptic regularity βf has to be smooth. This completes the proof of the proposition. Lemma 4.8. If λ ∈ ?ρ ? N0 , then the complex

P A

0 → Oλ C ?ω (?X ) → Oλ C ∞ (X ) → Oλ C ∞ (X ) → 0 is exact. Proof. Since P? is injective for ? ∈ ?ρ ? N0 the map P is injective. Exactness in the middle follows from Proposition 4.7. It remains to show that A is surjective. This turns out to be quite complicated. We will need the following technical result. Proposition 4.9. Let (M, g) be a connected, noncompact, real analytic, Riemannian manifold, and let ?M be the associated Laplace operator. Let r ∈ C and U ? C be open. Then the operator D : C ∞ (U × M ) → C ∞ (U × M ) given by (Df )(?, m) := ((?M + r + ?2 )f )(?, m) is surjective.

?∞ (U ×M ) → C ?∞ (U ×M ). Proof. We consider the adjoint operator t D : Cc c To prove surjectivity of D it is su?cient to show that t D is injective and has closed range. ?∞ (U × M ) satisfy First we prove that t D is injective. Let f ∈ Cc t Df = 0. We consider f as a hyperfunction in B U ×M (U × M ). The hyt perplanes {? = const} are non-characteristic for D . By a theorem of Sato [44], f (?, m) contains the variable m ∈ M as a real analytic parameter. Since supp(f ) is compact we conclude by [22, Thm. 1.5], that f = 0. This shows injectivity of t D . ?∞ (U × M ) be We now prove that the range of t D is closed. Let fi ∈ Cc t ?∞ a sequence such that Dfi =: hi converges to h ∈ Cc (U × M ). We have to ?∞ (U × M ) such that t Df = h. ?nd f ∈ Cc There is a compact subset T × W ? U × M such that supp(hi ) ? T × W ∞ (U ) such that χ t for all i. Let χ ∈ Cc |T ≡ 1. Since D is of order zero with

SELBERG ZETA FUNCTION

671

respect to the ?rst variable we have t D (χfi ) = χhi = hi . Replacing fi by χfi and enlarging T we can and will assume that supp(fi ) ? T × M . We will show that limi→∞ fi =: f exists. By enlarging W we can assume that any connected component of the closure of M \ W is noncompact. We claim that supp(fi ) ? T × W for all i. We will argue by contradiction. Let p : C × M → M denote the projection. Let V be a connected component of M \ W and assume that (fi )|U ×V = 0. Then by the unique continuation result [22, Thm. 1.5], p(supp(fi )) ? V . This is impossible since supp(fi ) and thus p(supp(fi )) are compact. It follows that (fi )|U ×V = 0 for all connected components V of M \ W . Let V ? M be an open subset with smooth boundary ?V such that ∞ (V ) with W ? V . We choose a real nonnegative potential 0 = F ∈ Cc supp(F ) ? V \ W . For small t we consider the Dirichlet extension ?t of the operator ?M + tF on L2 (V ). Then ?t is self-adjoint with eigenvalues 0 < κ1 (t) < κ2 (t) ≤ κ3 (t) ≤ . . . (counted with multiplicity) such that κj (t) → ∞ as j → ∞ for ?xed t. Furthermore, the eigenvalues κi are continuous, strictly increasing functions of t. We employ this fact in order to choose for any δ ∈ U a number tδ such that ?(r + δ2 ) ∈ spec(?tδ ). Let H s (V ) be the scale of Sobolev spaces associated with ?0 . For s ≥ 0 we have H s (V ) := dom(1 + ?0 )s/2 = dom(1 + ?t )s/2 . If s < 0, then H s (V ) := H ?s (V )′ . By elliptic regularity s∈R H s (V ) ? C ∞ (V ) and dually s ∞ ?∞ (V ) ? Cc s∈R H (V ). If χ ∈ Cc (V ), then multiplication by χ de?nes a continuous map χ : C ∞ (V ) → s∈R H s (V ). If d ∈ spec(?tδ ), then there is a neighbourhood Zδ ? C \ spec(?tδ ) of d such that Zδ ? a → (?tδ ? a)?1 : s∈R H s (V ) → s∈R H s (V ) is a holomorphic family of continuous isomorphisms. We choose open neighbourhoods Uδ of δ such that ?(r + ?2 ) ∈ Zδ for all ? ∈ Uδ . We choose a countable set of points δj ∈ U such that {Uδj } is a locally ?nite cover of U . Let {ξj } be an associated smooth partition of unity. Set ∞ (V ) ? C ∞ (M ) with χ Uj := Uδj , tj := tδj and choose χ ∈ Cc |W ≡ 1. Then we c de?ne continuous maps Lj : C ∞ (Uj × M ) → C ∞ (Uj ,

s∈R

H s (V )) ? C ∞ (Uj × V )

by Lj (φ)(?) := (?tj + r + ?2 )?1 χφ? , where φ? (m) = φ(?, m). Note that ?∞ (U × V ). Thus we can de?ne the distributions f? ∈ C ?∞ (U × M ) ξj hi ∈ Cc j i,j j c ∞ (U × V ); i.e., f? := t L (ξ h ). by f? , φ := ξ h , L ( φ ) , for all φ ∈ C i,j j i j j i,j j j i ∞ We claim that f? i,j = ξj fi . Indeed, we have for all φ ∈ C (Uj × M ) f? i,j , φ = = = ξj hi , Lj (φ) ξj t Dfi , Lj (φ)

t

D (ξj fi ), Lj (φ)

672

ULRICH BUNKE AND MARTIN OLBRICH

= = = =

(t D + tj F )(ξj fi ), Lj (φ) ξj fi , (?tj + r + ?2 )Lj (φ) ξj fi , χφ ξj f i , φ .

Consider the set I := {j ∈ N | Uj ∩ T = ?}. By local ?niteness of the cover {Uj } and compactness of T the set I is ?nite. By the claim above we t ? have f? i,j = 0 for j ∈ I and fi = j ∈I Lj (ξj hi ). Since hi → h as j ∈I fi,j = t i → ∞ and Lj is continuous we conclude that limi→∞ fi =: f exists. This ?nishes the proof of Proposition 4.9. We now ?nish the proof of Lemma 4.8, proving surjectivity of A : Oλ C ∞ (X ) → Oλ C ∞ (X ) . Let [f ] ∈ Oλ C ∞ (X ) be represented by f ∈ C ∞ (U × X ), where U is a precompact neighbourhood of λ with smooth boundary such that U ∩ ?ρ ? N0 = ? ? = 0. By Proposition 4.9 we can ?nd h ∈ C ∞ (U × X ) with Ah = f . and ?f ? ? . Thus ?h ? is a smooth family of eigenfunctions. By We have 0 = ?Ah = A?h ? ) ∈ C ∞ (U, C ?ω (?X )) such Proposition 4.7 there exists a smooth family β (?h ? ? )) = ?h ? . Here we have identi?ed C ω (?X, V (?)) with C ?ω (?X ), that P (β (?h as usual. ?? ?? on L2 (U ). Then ??1 is bounded Let ?U be the Dirichlet extension of ? U 1 ∞ ∞ and can be considered as a continuous operator ?? U : Cc (U ) → C (U ). Us∞ ? ω ∞ ? ω ? C (?X ) we obtain an induced oping that C(c) (U, C (?X )) = C(c) (U )? ?1 ? ? U be a neighbourhood of λ and erator Q = ?U ? id. Let W ? W ∞ ?? Q(χβ (?h ? )). Then choose χ ∈ Cc (U ) such that χ|W ≡ 1. We de?ne k := ? ? ? (?k)|W = (β (?h))|W . If we de?ne φ ∈ C ∞ (W × X ) by φ := (h ? P k)|W ×X , then ? =? ?(h ? P k)|W ×X = ?h ? |W ×X ? (P ? ?)k|W ×X = ?h ? |W ×X ? (P β ?h ? )|W ×X = 0 ?φ and Aφ = A(h ? P k)|W ×X = Ah|W ×X = f|W ×X . We let [φ] ∈ Oλ C ∞ (X ) be represented by φ and obtain A[φ] = [f ]. This proves surjectivity of A and thus Lemma 4.8. Lemma 4.10. The Γ-module Oλ C ∞ (X ) is acyclic.

Proof. If V ? X is open, then let Oλ C ∞ (V ) be the space of germs at λ of holomorphic functions with values in C ∞ (V ). The prescription X ? V → Oλ C ∞ (V ) de?nes a soft sheaf on X . Since Γ acts freely and properly on X , and the Γ-module Oλ C ∞ (X ) is the space of sections of a Γ-equivariant soft sheaf on X , we can argue as in the proof of [4, Lemma 2.4].

SELBERG ZETA FUNCTION

673

We return to the proof of Proposition 4.6. By Lemma 4.8 and Lemma 4.10 0 → Oλ C ∞ (X ) → Oλ C ∞ (X ) → 0 is a Γ-acyclic resolution of Oλ C ?ω (?X ). Since Oλ C ∞ (X ) = O?λ C ∞ (X ) we can assume that Re(λ) ≥ 0. Let Oλ C ∞ (Y ) := Γ Oλ C ∞ (X ) be the space of germs at λ of holomorphic functions with values in C ∞ (Y ) and AY : Oλ C ∞ (Y ) → Oλ C ∞ (Y ) be the operator induced by A. Then H 0 (Γ, Oλ C ?ω (?X ))

1 ?ω A

= ker(AY ) , = coker(AY ) , = 0, for all p≥2.

H (Γ, Oλ C

H (Γ, Oλ C

p

(?X ))

?ω

(?X ))

To prove Proposition 4.6 it remains to show that coker(AY ) = 0 or, equivalently, that AY is surjective. The proof of surjectivity of AY is similar to the proof of surjectivity of A. ? = 0. Let [f ] ∈ Oλ C ∞ (Y ) be represented by f ∈ C ∞ (U × Y ) satisfying ?f By Proposition 4.9 we can ?nd h ∈ C ∞ (U × Y ) with AY h = f . We have ? Y h = AY ?h ? . By p? : C ∞ (Y ) ?→ C ∞ (X ) we denote the inclusion 0 = ?A given by the pull-back associated to the covering projection p : X → Y . Then ? = 0 and β? (p? ?h ? )? ∈ Γ C ?ω (?X, V (?)) is de?ned. The family ? → Ap? ?h ? )? ∈ C ?ω (B, VB (?)) is a smooth family of analytic functionals. res? ? β? (p? ?h Using a suitable holomorphic trivialization of the family of bundles VB (?) ? ) ∈ we can identify C ?ω (B, VB (?)) with C ?ω (B ). We consider res ? β (p? ?h ∞ ? ω ? ? = C (U, C (B )). As in the proof of Lemma 4.8 we solve the ? -problem ?k ? ∞ ? ω ? ) for k ∈ C (W, C (B )), where W ? W ? ? U is a smaller res ? β (p ?h neighbourhood of λ. Replacing k by k ? k(λ), we can and will assume that k(λ) = 0. By our assumption Re(λ) ≥ 0 and Corollary 3.13, ext? is regular at ? = λ or it has a pole of at most ?rst order. Thus ext(k) is smooth on W \ {λ} and bounded on W . We de?ne the family W ? ? → φ? := h? ? P? ? ext? (k? ) ∈ C ∞ (Y ). This ? = 0. We conclude that family is bounded on W , and on W \ {λ} it satis?es ?φ ∞ φ is a holomorphic function from W to C (Y ). Moreover, AY φ = f|W ×Y . If we de?ne [φ] ∈ Oλ C ∞ (Y ) to be the element which is represented by φ, then AY [φ] = [f ]. This proves that AY is surjective. The proof of Proposition 4.6 is now complete. 4.3. Computation of H ? (Γ, Oλ C ?ω (Λ)).

Let Oλ C ?ω (B ) be the space of germs at λ of holomorphic families ? → f? ∈ C ?ω (B, VB (?)) and res : Γ Oλ C ?ω (?X ) → Oλ C ?ω (B ) be de?ned by res(f )? := res? f? .

674 Lemma 4.11. and (31)

ULRICH BUNKE AND MARTIN OLBRICH

If λ ∈ ?ρ ? N0 , then H p (Γ, Oλ C ?ω (Λ)) = 0 for all p = 1 H 1 (Γ, Oλ C ?ω (Λ)) ? = coker(res) .

Proof. By Lemmas 4.4, 4.5, and Proposition 4.6 the complex

? Oλ C ?ω (?) → 0 0 → Oλ C ?ω (?X ) →

res

is a Γ-acyclic resolution of Oλ C ?ω (Λ). If we employ the identi?cations

Γ

C ?ω (?, V (?)) = C ?ω (B, VB (?)) ,

then the corresponding complex of Γ-invariant vectors can be written in the form res 0 → Γ Oλ C ?ω (?X ) → Oλ C ?ω (B ) → 0 . We conclude that H (Γ, Oλ C

p

H 0 (Γ, Oλ C ?ω (Λ))

1 ?ω

= = =

ker(res) , coker(res) , 0, for all p ≥ 2 .

(Λ))

H (Γ, Oλ C

?ω

(Λ))

Since res? is injective for generic ? ∈ C we have ker(res) = 0 (see [8, Prop. 6.11]). The aim of this subsection is to express H p (Γ, Oλ C ?ω (Λ)) in terms of spectral and topological data as eigenvalues of the Laplacian, behaviour of the scattering matrix and cohomology of Γ with values in ?nite-dimensional representations. We start with the case Re(λ) ≥ 0. Proposition 4.12. p = 1 and H 1 (Γ, Oλ C ?ω (Λ)) ? = ? = If Re(λ) ≥ 0, then H p (Γ, Oλ C ?ω (Λ)) = 0 for all

Γ

C ?ω (Λ, V (λ)) dim kerL2 (?Y ? ρ2 + λ2 ) Re(λ) ≥ 0, λ = 0 dim ker(S0 + id) λ=0 .

In particular, if H 1 (Γ, Oλ C ?ω (Λ)) = 0, then λ ∈ F ? [0, δΓ ] (see Corollary 3.13). Proof. By Lemma 4.11 it remains to compute coker(res). Since extλ has at most ?rst order poles for Re(λ) ≥ 0 and res ? ext = id we can de?ne ev : Oλ C ?ω (B ) → Γ C ?ω (Λ, V (λ)) by ev (f ) := res?=λ ext? (fλ ). We claim that the sequence (32) 0 → Γ Oλ C ?ω (?X ) → Oλ C ?ω (B ) → Γ C ?ω (Λ, V (λ)) → 0

res ev

SELBERG ZETA FUNCTION

675

is exact. It is clear that res is injective. Surjectivity of ev follows from Lemma 3.12. Let f ∈ ker(ev ). Then ext(f ) ∈ Γ Oλ C ?ω (?X ) and res ? ext(f ) = f . It remains to show ev ? res = 0. Let φ ∈ Γ Oλ C ?ω (?X ). Then ext? ? res? (φ? ) = φ? , for all ? = λ, ? close to λ. It follows ev ? res(φ) = lim?→λ (? ? λ)ext? ? res? (φ? ) = lim?→λ (? ? λ)φ? = 0. This proves the claim. Exactness of (32) immediately implies coker(res) ? = Γ C ?ω (Λ, V (λ)). If Re(λ) > 0, then the Poisson transform provides an isomorphism between Γ C ?ω (Λ, V (λ)) and ker (? ? ρ2 + λ2 ). In fact, that P maps Γ C ?ω (Λ, V (λ)) Y λ L2 injectively into kerL2 (?Y ? ρ2 + λ2 ) was already observed in the proof of Lemma 2.13. For the surjectivity of Pλ see e.g. [8, Prop. 9.2] or [6, Lemma 2.1]. If Re(λ) = 0, λ = 0, then Γ C ?ω (Λ, V (λ)) as well as kerL2 (?Y ? ρ2 + λ2 ) are trivial (see Corollary 3.13 and [28], respectively). For a derivation of the latter fact in the framework of the present paper we refer to [8]. We ?nish the proof of the proposition by showing that Γ C ?ω (Λ, V (0)) ? = im(S0 ? id). From the proof of Lemma 3.11 we recall the equation S0 ? id = 0 0 ? res C res0 ? J0 ?=0 ext? , where C is some nonzero constant and J0 is the ?? at 0. Now by Lemma 3.12 we constant term in the Laurent expansion of J Γ ? ω 0 is injective by Lemma 3.7. have im(res?=0 ext? ) = C (Λ, V (0)), and res0 ? J0 0 provides the desired isomorphism. Thus res0 ? J0 In order to compute H ? (Γ, Oλ C ?ω (Λ)) for Re(λ) < 0 we need detailed information about the singularities of the intertwining operators. Lemma 4.13. (i) If λ ∈ ?ρ ? N0 , then the range of the Knapp-Stein in?λ is an irreducible ?nite-dimensional representation tertwining operator J Fλ of G. ?λ is an isomorphism. (ii) If Re(λ) < 0 and λ ∈ ?ρ ? N0 , then J ?λ has a pole if and only if λ ∈ N0 , and this pole is of ?rst order. (iii) J (iv) We consider the following renormalized versions of the intertwining opλ := (? ? λ)J ?? . If Re(λ) > 0 and λ ∈ N, then erators : If λ ∈ N, then J? λ λ ?? (thus J? is regular at ? = λ). J? := J

λ ) = ker(J ??λ ) and ker(J λ ) = im(J ??λ ). If If λ ∈ ρ + N0 , then im(Jλ λ λ Re(λ) > 0, λ ∈ ρ + N0 , then Jλ is an isomorphism. λ is de?ned by J λ (v) If λ ∈ ?ρ ? N0 , then the meromorphic family J? ? ? 1 λ ? ? := (? ? λ) J? . If Re(λ) < 0 and λ ∈ ?ρ ? N0 , then J? := J? .

If Re(λ) < 0, then there exists a holomorphic function q λ (?) which is λ ? J ?λ = de?ned in a small neighbourhood of λ such that q λ (λ) = 0 and J? ?? q λ (?)id.

676

ULRICH BUNKE AND MARTIN OLBRICH

Proof. The lemma is a consequence of several well-known facts concerning intertwining operators and reducibility of principal series representations. By [20, Ch.VI, Thm. 3.6], the principal series representation of G on ? ω C (?X, V (λ)) is (topologically) reducible if and only if λ ∈ ±(ρ + N0 ). If λ ∈ ρ + N0 , then by the Cartan-Helgason Theorem [19, Ch. V, Thm. 4.1], and Casselman’s Frobenius reciprocity [49, 3.8.2], C ?ω (?X, V (λ)) contains a ?nitedimensional irreducible submodule F?λ . If Re(λ) < 0, then by [49, 5.4.1.(2)], ?λ is exactly the unique irreducible submodule of C ?ω (?X, V (?λ)) the range of J (the “Langlands quotient”). If λ ∈ ?ρ ? N0 , then this submodule is Fλ . This proves (i). If Re(λ) < 0, λ ∈ ?ρ ? N0 , then C ?ω (?X, V (±λ)) are irreducible ?λ is an isomorphism. This is assertion (ii). and J (iii) follows from [26, Thm. 3 and Prop 4.4] (see also [20, Ch. II, Thm. 5.4]). In order to apply Prop. 4.4 (loc. cit.) we must know that P (0) = 0. One can either employ irreducibility of C ?ω (?X, V (0)) and [26, 7.1], or the explicit formulas for c(λ) (and the relation (14) P (λ)?1 = c(λ)c(?λ) ) given in [19, Ch. IV, Thm. 6.14]. We now consider (iv) and (v). If Re(λ) < 0, λ ∈ ?ρ ? N0 , then by the nonvanishing result [20, Ch. II, Prop. 5.7], and the irreducibility of ±λ C ?ω (?X, V (±λ)) both J± ? are isomorphisms for ? in a neighbourhood of ? λ λ λ λ. Thus J? ? J?? = q (?)id for some nowhere-vanishing local holomorphic function q λ . Let now λ ∈ ?ρ ? N0 . By (13), (33) where r (?) =

λ ?λ ?λ λ ?? ? J? ? J ? = J?? ? J? = r (?)id , 1 P (?) ??λ P (?)

n ≡ 0(2) n ≡ 1(2)

.

?λ λ ? Since J? λ is regular and Jλ is not surjective by (i), we obtain r (λ) = 0. We conclude that P has a pole at λ if n ≡ 0(2), and that P (λ) = 0 if n ≡ 1(2). By [26, Sec. 12], the Plancherel density P has at most simple poles if n ≡ 0(2), and it is holomorphic if n ≡ 1(2). Thus the zero of r λ at ? = λ is simple. Now (iv) follows from (33), and

q λ (?) :=

1 (??λ)P (?) 1 P (?)

n ≡ 0(2) n ≡ 1(2)

is holomorphic and nonvanishing in a neighbourhood of λ. This ?nishes the proof of (v). Fix λ ∈ ?ρ ? N0 . We de?ne the evaluation map b : Oλ C ?ω (?X ) → Fλ ?λ φλ . Set O0 C ?ω (?X ) := ker b. Let res0 be the restriction of by b(φ) := J λ 0 C ?ω (?X ). res : Γ Oλ C ?ω (?X ) → Oλ C ?ω (B ) to Γ Oλ

SELBERG ZETA FUNCTION

677

In order to make our notation more uniform we set for Re(λ) < 0, λ ∈ ? ρ ? N0 :

0 ?ω Fλ := 0, b := 0, res0 := res, Oλ C (?X ) := Oλ C ?ω (?X ) .

Then for all λ ∈ C with Re(λ) < 0 we have an exact complex of G-modules (34)

0 ?ω 0 → Oλ C (?X ) → Oλ C ?ω (?X ) → Fλ → 0 . b 0 C ?ω (?X ) → O ?ω (?X ) given by There is a well-de?ned map J λ : Oλ ?λ C λ f . It follows from Lemma 4.13 (v), that it is an isomorphism. (J λ f )?? := J? ? 0 C ?ω (?X ) is acyclic. Thus, by Proposition 4.6 the Γ-module Oλ

Lemma 4.14. (35)

If Re(λ) < 0, then for all p ≥ 1

H p (Γ, Oλ C ?ω (?X )) ? = H p (Γ, Fλ ) .

Proof. This is an immediate consequence of (34) and the Γ-acyclicity of

0 C ?ω (?X ). Oλ 0 C ?ω (B ) Next we introduce a regularized scattering matrix. Let O? λ 0 C ?ω (B ) = im(res). We de?ne := ker(ev ). Then by exactness of (32) O? λ 0 C ?ω (B ) → O C ?ω (B ) by S ?λ := res ? J ?λ ? ext. Note that S ?λ is S ?λ : O? λ ? λ regular at λ = ? since ext? is so when restricted to ker(ev ).

Lemma 4.15.

If Re(λ) < 0, then coker(res0 ) = coker(S ?λ ).

Proof. The maps

0 ?ω J λ : Γ Oλ C (?X ) → Γ O?λ C ?ω (?X )

and

0 ?ω res : Γ O?λ C ?ω (?X ) → O? (B ) λC ?λ ? ext (see Lemma are isomorphisms. The inverse of res ? J λ is given by q1 λJ 0 C ?ω (B )) = Γ O 0 C ?ω (?X ). We conclude that 4.13 (v)). Thus J ?λ ? ext(O? λ λ 0 ? λ coker(res ) = coker(S ).

Proposition 4.16.

If Re(λ) < 0, then

(i) H 0 (Γ, Oλ C ?ω (Λ)) = 0 , (ii) dim H 1 (Γ, Oλ C ?ω (Λ)) = dim coker(res) + dim H 1 (Γ, Fλ ) = dim coker(S ?λ )+dim H 1 (Γ, Fλ )?dim H 0 (Γ, Fλ ) , (iii) H p (Γ, Oλ C ?ω (Λ)) ? = H p (Γ, Fλ ) for all p ≥ 2 .

678

ULRICH BUNKE AND MARTIN OLBRICH

In particular, dim H ? (Γ, Oλ C ?ω (Λ)) < ∞ and χ(Γ, Oλ C ?ω (Λ)) = χ(Γ, Fλ ) ? dim coker(S ?λ ) . Proof. According to Lemma 4.14 the long exact cohomology sequence associated to (25) reads as follows: (36) 0 → Γ Oλ C ?ω (?X ) → Oλ C ?ω (B ) → (37) 0 → H p (Γ, Oλ C ?ω (Λ))

1 →

res

δ

b

→

bp

H p (Γ, Fλ ) → 0 ,

H 1 (Γ, Fλ ) → 0 ,

H 1 (Γ, Oλ C ?ω (Λ)) p≥2.

This implies (i), (iii) and the ?rst equation of (ii). The short exact sequence of complexes 0 → 0 →

Γ O 0 C ?ω (?X ) λ Γ O C ?ω (?X ) λ res0

↓i

→

res

→

Oλ C ?ω (B ) → coker(res0 ) → 0 ↓ ↓ ? ω Oλ C (B ) → coker(res) → 0

induces the exact sequence 0 → coker(i) → coker(res0 ) → coker(res) → 0 .

0 C ?ω (?X ) is acyclic we ?nd by (34) that coker(i) ? H 0 (Γ, F ). ComSince Oλ = λ bining this with (36) and Lemma 4.15 we obtain the remaining assertions.

4.4. The Γ-modules O(λ,k) C ?ω (Λ). De?nition 4.17. For any k ∈ N we de?ne the Γ-module O(λ,k) C ?ω (Λ) as the quotient: (38)

λ 0 → Oλ C ?ω (Λ) → Oλ C ?ω (Λ) → O(λ,k) C ?ω (Λ) → 0 ,

Lk

k k where Lk λ is de?ned by (Lλ f )? := (? ? λ) f? . For any λ ∈ C de?ne k (λ) := Ord?=λ ext? + ε(λ), where ε(λ) = 0 if λ ∈ ?ρ ? N0 , and ε(λ) = 1 elsewhere. Here Ord?=λ denotes the (positive) order of a pole at ? = λ, if there is one, and zero otherwise.

There are isomorphisms of Γ-modules O(λ,1) C ?ω (Λ) ? = C ?ω (Λ, V (λ)). If Re(λ) ≥ 0, then by Corollary 3.13 we have k(λ) ≤ 1. We consider the operators

p ?ω (Lk (Λ)) → H p (Γ, Oλ C ?ω (Λ)) λ )p : H (Γ, Oλ C

induced by Lk λ. Lemma 4.18. If p = 1, then (Lλ )p = 0. If k ≥ k(λ), then (Lk λ )1 = 0.

SELBERG ZETA FUNCTION

679

Proof. Because of the triviality of H 0 (Γ, Oλ C ?ω (Λ)) it is enough to consider the case p > 0. If λ ∈ ?ρ ? N0 , then H p (Γ, Oλ C ?ω (Λ)) = 0 for all p = 1 by Lemma 4.11; thus (Lλ )p = 0. Let λ ∈ ?ρ ? N0 . We give Fλ the structure of a C[Lλ ]-module setting Lλ v := 0, v ∈ Fλ . Then b becomes a morphism of Γ- and C[Lλ ]-modules. Thus (37) is an isomorphism of C[Lλ ]-modules. Hence (Lλ )p = 0 for p ≥ 2. Consider (25) as an exact sequence of Γ- and C[Lλ ]-modules. Then (31) and (36) become sequences of C[Lλ ]-modules. Let λ ∈ C with λ ∈ ?ρ ? N0 . If [f ] ∈ H 1 (Γ, Oλ C ?ω (Λ)) ? = coker(res) is represented by f ∈ Oλ C ?ω (B ), then k Γ ? ω ext ? Lλ f =: g ∈ Oλ C (?X ) exists for k ≥ Ord?=λ ext? = k(λ). It follows k that (Lk λ )1 [f ] = [Lλ f ] = [res(g )] = 0 for k ≥ k (λ). Let now λ ∈ ?ρ ? N0 . Consider φ ∈ H 1 (Γ, Oλ C ?ω (Λ)). We employ the sequence (36). Since (Lλ )1 acts trivially on H 1 (Γ, Fλ ) we have b1 ? (Lλ )1 (φ) = (Lλ )1 ? b1 (φ) = 0. Thus (Lλ )1 φ = δ(f ) for some f ∈ Oλ C ?ω (B ). Suppose k ?1 )1 (f ) we obtain that k ≥ Ord?=λ ext? + 1 = k(λ). Putting g := ext ? (Lλ k (Lλ )1 φ = δ ? res(g) = 0. This ?nishes the proof of the lemma. We recall the de?nition of the ?rst derived Euler characteristic χ1 (Γ, V ) of a Γ-module V :

∞

χ1 (Γ, V ) :=

p=0

(?1)p p dim H p (Γ, V ) .

The following proposition contains the ?rst three assertions of Theorem 1.3 and the fact that equation (3) implies equation (4). Proposition 4.19. Let λ ∈ C. Then

(i) dim H ? (Γ, O(λ,k) C ?ω (Λ)) < ∞. In particular, dim H ? (Γ, C ?ω (Λ, V (λ))) < ∞. (ii) χ(Γ, O(λ,k) C ?ω (Λ)) = 0. If k ≥ k(λ), then (iii) dim H ? (Γ, O(λ,k+1) C ?ω (Λ)) = dim H ? (Γ, O(λ,k) C ?ω (Λ)). (iv) χ1 (Γ, O(λ,k) C ?ω (Λ)) = χ(Γ, Oλ C ?ω (Λ)). Proof. Assertions (i) and (ii) follow from Proposition 4.16 and the long exact cohomology sequence associated to (38). If k ≥ k(λ), then by Lemma 4.18 this long exact sequence splits into short exact sequences (p = 0, 1, . . .), 0 → H p (Γ, Oλ C ?ω (Λ)) → H p (Γ, O(λ,k) C ?ω (Λ)) → H p+1 (Γ, Oλ C ?ω (Λ)) → 0 . Now assertions (iii) and (iv) follow, too.

680

ULRICH BUNKE AND MARTIN OLBRICH

Proposition 4.20.

χ(Γ, Oλ C

?ω

If λ ∈ C and k ≥ k(λ), then

=

(Λ))

χ1 (Γ, O(λ,k) C ?ω (Λ))

? 2 2 ? ? dim kerL2 (?Y ? ρ + λ ) ?

? dim ker(S0 + id)

?λ

Re(λ) ≥ 0, Re(λ) < 0 .

λ=0 λ=0

? dim coker(S

) + χ(Γ, Fλ )

Proof. The proposition is an immediate consequence of Propositions 4.12, 4.16 and 4.19. In fact, a closer study of the long exact cohomology sequence associated to (38) in combination with Propositions 4.12 and 4.16 gives more precise information about the cohomology groups H ? (Γ, O(λ,k) C ?ω (Λ)). Corollary 4.21. If Re(λ) ≥ 0, then set Fλ := 0. For all λ ∈ C and k ∈ N we have the following isomorphisms and exact sequences :

1 ?ω 0 → coker(Lk (Λ)) λ )1 → H (Γ, O(λ,k ) C

H 0 (Γ, O(λ,k) C ?ω (Λ))

? =

ker(Lk λ )1 , H p+1 (Γ, Fλ ) → 0 , H 2 (Γ, Fλ ) → 0 , p≥2.

0 → H p (Γ, Fλ ) → H p (Γ, O(λ,k) C ?ω (Λ))

→

→

In particular, dim H 1 (Γ, O(λ,k) C ?ω (Λ)) = dim H 0 (Γ, O(λ,k) C ?ω (Λ)) + dim H 2 (Γ, Fλ ) and if k ≥ k(λ), then

dim H 0 (Γ, O(λ,k) C ?ω (Λ)) = dim coker(res) + dim H 1 (Γ, Fλ )

=

? 2 2 Γ ?ω ? dim C (Λ, V (λ)) = dim kerL2 (?Y ? ρ + λ ) ?

dim C

Γ ?ω

Re(λ) ≥ 0, Re(λ) < 0 .

λ=0 λ=0

(Λ, V (λ)) = dim ker(S0 + id)

dim coker(S ?λ ) + dim H 1 (Γ, Fλ ) ? dim H 0 (Γ, Fλ )

We conclude this subsection with a generalization of Lemma 3.12 to the case λ ∈ ?ρ ? N0 . For k ≥ k(λ) the singular part of ext at λ de?nes a map 0 ?ω (B ) → Γ O ?ω (Λ) by ext< (λ,k ) C λ : Oλ C

0 k k ?ω ext< (?X )) . λ (f ) := ext ? Lλ (f ) mod Lλ (Oλ C

Proposition 4.22.

If λ ∈ ?ρ ? N0 and k ≥ k(λ), then

0 ?ω ext< (B ) → Γ O(λ,k) C ?ω (Λ) λ : Oλ C

is surjective. Moreover,

Γ

C ?ω (Λ, V (λ)) =

res?=λ (ext? (f? )) |

f ∈ Oλ C ?ω (B ) such that ext? (f? ) has a pole of ?rst order at ? = λ

.

In particular, ext? is regular at λ if and only if Γ C ?ω (Λ, V (λ)) = 0.

SELBERG ZETA FUNCTION

681

0 Proof. ext< λ factorizes over coker(res), and since res ? ext = id, this factorization is injective. By Corollary 4.21, dim coker(res) = dim Γ O(λ,k) C ?ω (Λ). 0 This implies surjectivity of ext< λ . Again by Corollary 4.21 it follows that

dim Γ C ?ω (Λ, V (λ)) = dim ker(Lλ : coker(res) → coker(res)) . The remaining assertions of the proposition are now obvious. Remark. Since in general for λ ∈ ?ρ ? N0 we have dim H 1 (Γ, Fλ ) = 0, 0 the map ext< λ is not surjective in view of the formula dim Γ O(λ,k) C ?ω (Λ) = dim coker(res) + dim H 1 (Γ, Fλ ) .

5. The singularities of the Selberg zeta function 5.1. The embedding trick. Let ZS (λ) denote the Selberg zeta function associated to Γ introduced in subsection 1.1. For dim(X ) even the spectral description of its singularities was worked out in Patterson-Perry [38]. This description simpli?es considerably if we assume that δΓ < 0. We are going to prove the remaining assertion (iv), equation (3), of Theorem 1.3 in two steps: (i) First we employ the embedding trick (which was already used in the proof of Proposition 2.22) in order to show in Corollary 5.5 that the equality (3) under the additional assumptions δΓ < 0 and dim(X ) ≡ 0(2) implies (3) in general. (ii) Then we prove (3) under the additional assumptions δΓ < 0 and dim(X ) ≡ 0(2). In the present subsection we are concerned with step (i). We adopt the notation Gn , ?X n , etc. as introduced in the proof of Proposition 2.22. Proposition 5.1. (39) χ(Γ, Oλ C ?ω (Λn )) = χ(Γ, Oλ? 1 C ?ω (Λn+1 )) ? χ(Γ, Oλ+ 1 C ?ω (Λn+1 )) .

2 2

Proof. We will construct an exact sequence of Γ-modules (40)

? 0 → Oλ C ?ω (Λn ) → Oλ? 1 C ?ω (Λn+1 ) → Oλ+ 1 C ?ω (Λn+1 ) → 0 . 2 2

i

Φ

682

ULRICH BUNKE AND MARTIN OLBRICH

Here i? is the push forward associated to the embedding ?X n ?→ X n+1 (see the proof of 2.22). The map Φ is multiplication by a Gn -invariant real analytic section φ of V (ρn+1 + 1)n+1 → ?X n+1 which vanishes on ?X n of ?rst order. We will construct φ in Lemma 5.2 and show exactness of (40) in Lemma 5.3 below. Then Proposition 5.1 will follow in view of the long exact sequence in cohomology associated to (40) and the fact that the cohomology groups are ?nite-dimensional. Lemma 5.2. satis?es : There exists a real analytic function φ ∈ C ω (Gn+1 ) which

(i) {φ = 0} = Gn M n+1 An+1 N n+1 , (ii) φ(hg) = φ(g), for all h ∈ Gn , g ∈ Gn+1 , (iii) φ(gman) = aφ(g), for all man ∈ M n+1 An+1 N n+1 . (iv) If φ is considered as a section of V (ρn+1 + 1)n+1 , then it vanishes on ?X n of ?rst order. Proof. We consider the standard representation π of Gn+1 = SO(1, n + 1)0 on Rn+2 . Let {ei | i = 0, . . . , n + 1} be the standard base and ., . be the Gn+1 -invariant bilinear form diag(?1, 1, . . . , 1). We specify the Iwasawa decomposition by K n+1 := {g ∈ Gn+1 | ge0 = e0 } ? = SO(n + 1) and A+

?? ? ? cosh(t) sinh(t) ? := ? sinh(t) cosh(t) ? ? 0 0 ? 0 ? ? 0 ? |t>0 . ? ? idRn ? ?

Then we de?ne φ(g) := en+1 , π (g)(e0 + e1 ) . Since en+1 is Gn -invariant, e0 + e1 is M n+1 -invariant and the highest weight vector with respect to A+ , the assertions (ii) and (iii) follow. Thus we can consider φ as a Gn -invariant section of V (ρn+1 + 1)n+1 . In the usual trivialization by the K n+1 -invariant section it is simply given by the height function φ(x) = xn+1 , x = (x1 , . . . , xn+1 ) ∈ S n = ?X n+1 . This is easily seen using the K n+1 -equivariant embedding S n ? x → (1, x) ∈ Rn+2 . We conclude that {x ∈ ?X n+1 | φ(x) = 0} = {x ∈ ?X n+1 | xn+1 = 0} = ?X n , and that φ vanishes on ?X n of ?rst order. We now de?ne Φ? : C ω (?X n+1 , V (?)n+1 ) → C ω (?X n+1 , V (? + 1)n+1 ) by Φ? (f )(g) := φ(g)f (g). Indeed by Lemma 5.2 (iii), Φ? (f )(gman) = φ(gman)f (gman) = aφ(g)a??ρ Gn+1 , M n+1 An+1 N n+1 ,

n+1

f (g) = a?+1?ρ

n+1

Φ? (f )(g) ,

for all g ∈ man ∈ and thus Φ(f ) is a section of n +1 V (? + 1) . By Lemma 5.2 (ii), the map Φ? is Gn -equivariant. Φ? extends to hyperfunction sections and satis?es supp(Φ? f ) ? supp(f ), for all f ∈ C ?ω (?X n+1 , V (?)n+1 ).

SELBERG ZETA FUNCTION

683

We de?ne Φ : Oλ? 1 C ?ω (Λn+1 ) → Oλ+ 1 C ?ω (Λn+1 ) by (Φf )? := Φ? f? .

2 2

Lemma 5.3. The sequence (40) is exact. Proof. Using Lemmas 4.2 and 4.3 we obtain the following diagram:

0 ↓ Oλ C ?ω (?X n ) ↓ Oλ C ?ω (?n ) ↓ 0 0 ↓

? →

0 ↓ → →

Φ Φ

0 0

→ →

i

→

i?

Oλ? 1 C ?ω (?X n+1 ) 2 ↓ Oλ? 1 C ?ω (?n+1 ) 2 ↓ 0

Oλ+ 1 C ?ω (?X n+1 ) 2 ↓ Oλ+ 1 C ?ω (?n+1 ) 2 ↓ 0

→ →

0 , 0

where the rows are exact, and the columns are surjective. The sequence (40) is just the complex of kernels of the columns of the diagram above, and this complex is exact by the Snake lemma. This ?nishes the proof of both the lemma and Proposition 5.1. The signi?cance of Proposition 5.1 is that the numbers χ(Γ, Oλ C ?ω (Λn )) behave in the same way with respect to embeddings Gn ?→ Gn+1 as the orders of the singularities of the Selberg zeta function ZS . Let Γ ? Gn be convex cocompact. If we consider Γ as a convex cocompact subgroup of Gn+1 , then ZS,n+1 (s) denotes the corresponding Selberg zeta function. The following elementary fact was noted in [34]. Lemma 5.4. ZS,n+1 (s) = and, consequently,

∞

1 ZS,n (s + j + ) 2 j =0

∞

(41)

ords=λ ZS,n+1 (s) ords=λ ZS,n (s)

=

j =0

ords=λ+j + 1 ZS,n (s) ,

2

=

ords=λ? 1 ZS,n+1 (s) ? ords=λ+ 1 ZS,n+1 (s) .

2 2

Combining equation (41) with Proposition 5.1 we obtain: Corollary 5.5. Let Γ ? Gn be convex cocompact. If equation (3) holds true for Γ viewed as a subgroup of Gn+1 , then so does it for Γ viewed as a subgroup of Gn . In particular, it is su?cient to prove (3) under the assumptions δΓ < 0 and dim(X ) ≡ 0(2). 5.2. Singularities and cohomology. In this subsection we prove Theorem 1.3 (iv), (3), under the assumptions that n ≡ 0(2) and δΓ < 0. In [38] the order of the singularities of the Selberg

684

ULRICH BUNKE AND MARTIN OLBRICH

zeta function was expressed in terms of traces of residues of certain meromorphic families of operators. Our main task is to link this description with our computation of χ(Γ, Oλ C ?ω (Λ)). First observe that (3) holds true for Re(λ) ≥ 0. Indeed because of our assumption δΓ < 0 on the one hand the in?nite product (2) de?ning the Selberg zeta function ZS (λ) converges and thus ord?=λ ZS (?) = 0. On the other hand χ(Γ, Oλ C ?ω (Λ)) = 0 by Proposition 4.12. It remains to prove (3) for Re(λ) < 0. First we give an expression for 0 C ?ω (B ) → O C ?ω (B )) in terms of the trace of the dim coker(S ?λ : O? λ λ residue of the logarithmic derivative of S ?λ . Because of the assumption δΓ < 0 0 C ?ω (B ) = O ?ω (B ). we have O? ?λ C λ In the following we review results of Patterson-Perry [38]. We ?x an analytic Riemannian metric on B in the canonical conformal class. This metric induces a volume form which we employ in order to identify all bundles VB (λ) with B × C = VB (ρ). Then the scattering matrix becomes a germ at λ of a ?λ on C ? (B ), ? = ±ω, ±∞. meromorphic family of operators S? Let ?B be the Laplace operator on B associated to the Riemannian metric. Viewed as an unbounded symmetric operator on the Hilbert space H := L2 (B ) √ the sum ?B + 1 is positive. Let P := ?B + 1 be the positive square root de√ ?ned by spectral theory. Then P := ?B + 1 is a pseudodi?erential operator of order 1. In particular, P and its complex powers P ? act on C ±∞ (B ). For ? close to λ the scattering matrix can be factorized as (42)

?λ ?? (id + K (??))P ?? , S? ? =P

where K (??) is a holomorphic family of pseudodi?erential operators belonging to the (n + 1)th Schatten class (i.e. K (??)n+1 is of trace class). The inverse (1 + K (??))?1 is a meromorphic family of operators with ?nite-dimensional residues. Proposition 5.6. dim coker S ?λ : O?λ C ?ω (B ) → Oλ C ?ω (B ) = Tr res?=?λ (1+K (?))?1 K ′ (?) , where K ′ (?) denotes the derivative of K (?) with respect to ?. Proof. Let Oλ H denote the space of germs at λ of holomorphic families of vectors in H and let (1 + K ) : O?λ H → Oλ H be given by ((1 + K )f )? := (1 + K (??))f?? , f ∈ O?λ H. Lemma 5.7. dim coker(S ?λ ) = dim coker(1 + K ).

SELBERG ZETA FUNCTION

685

Proof. For k ∈ N we de?ne

?ω O(λ,k) C ?ω (B ) := coker(Lk (B ) → Oλ C ?ω (B )) . λ : Oλ C

Let S (?λ,k) : O(?λ,k) C ?ω (B ) → O(λ,k) C ?ω (B ) be the operator induced by 1 λ ?ω (B ) is S ?λ . Fix k ≥ k(λ). If f ∈ im(Lk λ ), then g := q λ S (f ) ∈ O?λ C de?ned and f = S ?λ (g) (see Lemma 4.13 (v)). It follows that Lk λ acts trivially on coker(S ?λ ). Hence we can identify coker(S ?λ ) with coker(S (?λ,k) ) in the natural way. As spaces of ?nite Taylor series with values in a Fr? echet space, the spaces O(±λ,k) C ?ω (B ) are Fr? echet spaces, too. By Lemma 2.20 the map S (?λ,k) is continuous with ?nite-dimensional cokernel. Hence it has closed range by the open mapping theorem. Thus the induced topology on coker(S ?λ ) = coker(S (?λ,k) ) is Hausdor?. Let Oλ C ?∞ (B ) denote the space of germs at λ of holomorphic families of distributions on B , and let O(λ,k) C ?∞ (B ) and O(λ,k) H be the corresponding quotient spaces of ?nite Taylor series. We de?ne P : Oλ C ?∞ (B ) → Oλ C ?∞ (B ) by (P f )? = P ?? f? . We denote the induced operator on O(λ,k) C ?∞ (B ) by the same symbol. The composition p : O(λ,k) H ?→ O(λ,k) C ?∞ (B ) → O(λ,k) C ?∞ (B ) ?→ O(λ,k) C ?ω (B ) has dense range. Now let h ∈ Oλ H. We claim that h ∈ im(1 + K ) if and only if P h considered as a hyperfunction is in the image of S ?λ . Indeed, let P h = S ?λ j for some j ∈ Oλ C ?ω (B ). Then ?? → g?? := (1 + K (?))?1 (h? ) de?nes a meromorphic family of vectors in H. It is regular at ? = λ since P ? (g?? ) = j?? by (42) and P ? is injective. Vice versa, let h = (1 + K )g for some g ∈ O?λ H. The family j?? := P ? (g?? ), ? near λ, de?nes an element j ∈ O?λ C ?∞ (B ) satisfying P h = S ?λ j . The above claim implies that p induces a map p? : coker(1 + K ) → coker(S ?λ ) which is injective. Moreover, it has dense range. Since coker(S ?λ ) is ?nite-dimensional and Hausdor? p? must be surjective. The lemma follows. The following lemma is known in one form or another (compare [15]). For completeness we include a proof here. Lemma 5.8. Tr res?=?λ (1 + K (?))?1 K ′ (?) = dim coker(1 + K ) .

P

686

ULRICH BUNKE AND MARTIN OLBRICH

Proof. Set s := ?λ. Let P (?) be the holomorphic family of ?nite-dimensional projections given by P (?) := 1 2π?

c

where the path of integration is a small circle enclosing 0 ∈ C counterclockwise and ? is close to s. There is a holomorphic family of invertible operators U (?) ([43, Thm. XII.12]) such that U (?)?1 P (?)U (?) = P (s). We de?ne T (?) := U (?)?1 (1 + K (?))U (?). Then T (?)P (s) = = U (?)?1 (1 + K (?))P (?)U (?) U (?)?1 P (?)(1 + K (?))U (?) = P (s)T (?) .

1 dz , z ? 1 ? K (?)

Let V := P (s)H and W := (1 ? P (s))H. Then T (?) = A(?) 0 0 B (?) V V : ⊕ → ⊕ , W W

where B (?) is invertible for ? ? s small. We claim that Tr res?=s T (?)?1 T ′ (?) = Tr res?=s (1 + K (?))?1 K ′ (?) . In fact T ′ (?) = U (?)?1 (1 + K (?))U ′ (?) + U (?)?1 K ′ (?)U (?) ?U (?)?1 U ′ (?)U (?)?1 (1 + K (?))U (?) . Using the facts that all singular terms of (1 + K (?))?1 are ?nite-dimensional and that the trace is cyclic, we compute Tr res?=s T (?)?1 T ′ (?) = Tr res?=s U (?)?1 (1 + K (?))?1 U (?) [U (?)?1 (1 + K (?))U ′ (?) + U (?)?1 K ′ (?)U (?) = Tr res?=s ?U (?)?1 U ′ (?)U (?)?1 (1 + K (?))U (?)]

[U (?)?1 U ′ (?) + U (?)?1 (1 + K (?))?1 K ′ (?)U (?) = Tr res?=s (1 + K (?))?1 K ′ (?) . ?U (?)?1 (1 + K (?))?1 U ′ (?)U (?)?1 (1 + K (?))U (?)]

This proves the claim. Let T : Os H → Os H be given by (T f )(?) = T (?)f (?), f ∈ Os H. Then we have dim coker(T ) = dim coker(1 + K ) .

SELBERG ZETA FUNCTION

687

Now Tr res?=s T (?)?1 T ′ (?) = Tr res?=s A(?)?1 A′ (?) for the holomorphic family of operators A(?) on the ?nite-dimensional space V . Moreover dim coker(T ) = dim coker(A), where A : Os V → Os V is given by (Af )(?) := A(?)f (?), f ∈ Os V . In order to ?nish the proof of the proposition we must show that Tr res?=s A(?)?1 A′ (?) = dim coker(A) . Now Tr res?=s A(?)?1 A′ (?) = = = res?=s Tr A(?)?1 A′ (?) det(A(?))′ res?=s det(A(?)) ord?=s det(A(?)) .

By Gauss’s algorithm A(?) can be transformed to a holomorphic family of di?(?) through multiplication from the left and right with holoagonal matrices A ?) morphic matrix functions with invertible determinants. We have dim coker(A ? ? ? = dim coker(A), where A : Os V → Os V is given by (Af )(?) := A(?)f (?), ?(?)) = ord?=s det(A(?)). But for holomorphic f ∈ Os V , and ord?=s det(A ? = ord?=s det(A ?(?)) is families of diagonal matrices the equation dim cokerA obvious. This ?nishes the proof of the lemma. Proposition 5.6 follows from Lemmas 5.7 and 5.8. We now recall the description of the singularities of the Selberg zeta function ZS (λ) for Re(λ) < 0 given in [38]. Note that our standing hypothesis is n ≡ 0(2) and δΓ < 0. This simpli?es things considerably because the point spectrum of ?Y is absent. Let Re(λ) < 0 and set nλ := Tr res?=?λ (1 + K (?))?1 K ′ (?). Then ords=λ ZS (s) = nλ λ ∈ ? ρ ? N0 nλ ? χ(Y ) dim Fλ λ ∈ ?ρ ? N0 .

Since Y has the homotopy type of a ?nite CW-complex we have χ(Y, Fλ ) = χ(Y ) dim Fλ . Equation (3) now follows from Propositions 5.6 and 4.20. By Corollary 5.5 a proof of (3) under the assumptions n ≡ 0(2) and δΓ < 0 implies (3) without these assumptions. This ?nishes the proof of Theorem 1.3.

¨ t Go ¨ ttingen, Go ¨ ttingen, Germany Universita E-mail address : bunke@uni-math.gwdg.de E-mail address : olbrich@uni-math.gwdg.de

688

ULRICH BUNKE AND MARTIN OLBRICH References

[1] S. Agmon, Spectral theory of Schr¨ odinger operators on Euclidean and non-Euclidean spaces, Comm. Pure Appl. Math 39 (1986), S3–S16. [2] E. P. van den Ban and H. Schlichtkrull, Asymptotic expansions and boundary values of eigenfunctions on Riemannian symmetric spaces, J. reine angew. Math. 380 (1987), 108–165. [3] , Local boundary data of eigenfunctions on a Riemannian symmetric space, Invent. Math. 98 (1989), 639–657. [4] U. Bunke and M. Olbrich, Gamma-cohomology and the Selberg zeta function, J. reine angew. Math. 467 (1995), 199–219. [5] , Selberg Zeta and Theta Functions, Akademie Verlag, 1995. [6] , Fuchsian groups of the second kind and representations carried by the limit set, Inv. Math. 127 (1997), 127–154. , Resolutions of distribution globalizations of Harish-Chandra modules and co[7] homology, J. reine angew. Math. 497 (1998), 47–81. , The spectrum of Kleinian manifolds. Preprint available at http://www.uni[8] math.gwdg.de/bunke/spzerl.dvi. [9] Y. Colin de Verdi? ere, Th? eorie spectrale et surfaces de Riemann d’aire in?nie, Asterisque 132 (1985), 259–275. [10] J. Elstrodt, Die Resolvente zum Eigenwertproblem der automorphen Formen in der hyperbolischen Ebene III, Math. Ann. 208 (1975), 99–132. [11] A. Erd? elyi, W. Magnus, F. Oberhettinger, and F. G. Tricomi, Higher Transcendental Functions, Vol. I, McGraw-Hill Book Co., New York, 1953. [12] J. D. Fay, Fourier coe?cients of the resolvent for a Fuchsian group, J. reine angew. Math. 293 (1977), 143–203. ? Norm. Sup. 4e [13] D. Fried, The zeta functions of Ruelle and Selberg I, Ann. Scient. Ec. S? erie 19 (1986), 491–517. [14] R. Gangolli, Zeta functions of Selberg’s type for compact space forms of symmetric spaces of rank one, Ill. J. Math. 21 (1977), 1–42. [15] I. C. Gohberg and E. I. Sigal, An operator generalization of the logarithmic residue theorem and a theorem of Rouch? e, Math. USSR Sbornik 13 (1971), 603–624. [16] V. Guillemin, Lectures on spectral theory of elliptic operators, Duke Math. J. 44 (1977), 485–517. [17] L. Guillop? e, Fonctions z? eta de Selberg et surface de g? eom? etrie ?nie, Adv. Stud. Pure Math. 21 (1992), 33–70. [18] S. Helgason, Eigenspaces of the Laplacian, integral representations and irreducibility, J. Funct. Anal. 17 (1974), 328–353. [19] , Groups and Geometric Analysis, Academic Press, Inc., 1984. [20] , Geometric Analysis on Symmmetric Spaces, Math. Surveys 39, A.M.S. Providence, R. I., 1994. [21] A. Juhl, Zeta-Funktionen, Index-Theorie und hyperbolische Dynamik, Habilitationsschrift, Humboldt-Universit¨ at zu Berlin, 1993. [22] A. Kaneko, Remarks on hyperfunctions with analytic parameters, J. Fac. Sci. Univ. Tokyo 22 (1977), 371–403. [23] M. Kashiwara, A. Kowata, K. Minemura, K. Okamoto, T. Oshima, and M. Tanaka, Eigenfunctions of invariant di?erential operators on a symmetric space, Ann. of Math. 107 (1978), 1–39. [24] M. Kashiwara and P. Schapira, Sheaves on Manifolds, Springer-Verlag, 1990. [25] A. W. Knapp, Representation Theory of Semisimple Lie Groups. An Overview Based on Examples, Princeton University Press, 1986. [26] A. W. Knapp and E. M. Stein, Intertwining operators for semisimple Lie groups, Ann. of Math. 93 (1971), 489–578.

SELBERG ZETA FUNCTION

689

[27] H. Komatsu, Relative cohomology of sheaves of solutions of di?erential equations. In Hyperfunctions and Pseudo-Di?erential Equations, Proc. Conf. at Katata 1971, LNM 287, pages 192–261, Springer-Verlag Berlin Heidelberg New York, 1973. [28] P. D. Lax and R. S. Phillips, The asymptotic distribution of lattice points in Euclidian and non-Euclidian spaces, J. Funct. Anal. 46 (1982), 280–350. [29] , Translation representations for automorphic solutions of the wave equation in non-Euclidian spaces I/II, Comm. Pure Appl. Math. 37 (1984), 303–328/779–813. [30] N. Mandouvalos, Spectral theory and Eisenstein series for Kleinian groups, Proc. London Math. Soc. 57 (1988), 209–238. , Scattering Operator, Eisenstein Series, Inner Product Formula and “Maass[31] Selberg ” Relations for Kleinian Groups, Memoirs A.M.S. 78, 1989. [32] R. Mazzeo and R. Melrose, Meromorphic extension of the resolvent on complete spaces with asymptotically constant negative curvature, J. Funct. Anal. 75 (1987), 260–310. [33] H. Nishimura, Heyting-valued set theory and Sato hyperfunctions, Publ. RIMS, Kyoto Univ. 22 (1986), 801–811. [34] S. J. Patterson, The Selberg zeta-function of a Kleinian group. In Number Theory, Trace Formulas and Discrete Groups: Symposium in Honor of Atle Selberg, Oslo, 1987, pages 409–441. [35] , The Laplace operator on a Riemann surface II, Comp. Math. 32 (1976), 71–112. [36] , On Ruelle’s zeta-function, Israel Math. Conf. Proc. 3 (1990), 163–184. [37] , Two conjectures on Kleinian groups, talk at Warwick, March, 1993. [38] S. J. Patterson and P. A. Perry, Divisor of the Selberg zeta function, I. Even dimension, submitted to Duke Math. J., 1996. [39] P. A. Perry, The Laplace operator on a hyperbolic manifold I. Spectral and scattering theory, J. Funct. Anal. 75 (1987), 161–187. , The Laplace operator on a hyperbolic manifold II. Eisenstein series and the [40] scattering matrix, J. Reine angew. Math. 398 (1989), 67–91. [41] , The Selberg zeta function and a local trace formula for Kleinian groups, J. Reine Angew. Math., 410 (1990),116–152. , Divisor of the Selberg zeta function for Kleinian groups, J. Equ. Deriv. Partielles [42] (St. Jean de Monts) 8 (1994), 9. [43] M. Reed and B. Simon, Methods of Modern Mathematical Physics IV. Analysis of Operators, Academic Press, Inc., 1978. [44] M. Sato, T. Kawai, and M. Kashiwara, Microfunctions and pseudo-di?erential equations. In Hyperfunctions and Pseudo-Di?erential Equations, Proc. Conf. at Katata 1971, LNM 287, pages 265–529, Springer-Verlag Berlin Heidelberg New York, 1973. [45] H. Schlichtkrull, Hyperfunctions and Harmonic Analysis on Symmetric Spaces, Birkh¨ auser, Boston, 1984. [46] W. Schmid, Boundary value problems for group invariant di?erential equations, Soc. Math. de France, Ast? erisque, hors s? erie, pages 311–321, 1985. [47] D. Sullivan, The density at in?nity of a discrete group of hyperbolic motions, I.H.E.S. Publ. Math. 50 (1979), 171–209. [48] M. Wakayama, Zeta functions of Selberg’s type associated with homogeneous vector bundles, Hiroshima Math. J. 15 (1985), 235–295. [49] N. R. Wallach, Real Reductive Groups, Academic Press, 1988. [50] , Real Reductive Groups II, Academic Press, 1992. (Received July 22, 1997)

- The Kazhdan property of the mapping class group of closed surfaces and the first cohomology
- Dedekind's-Function and the Cohomology of Infinite Dimensional Lie Algebras
- Poles of the topological zeta function associated to an ideal in dimension two
- The Divisor of Selberg's Zeta Function for Kleinian Groups
- The Selberg trace formula and Selberg zeta-function for cofinite Kleinian groups with finit
- Two complete and minimal systems associated with the zeros of the Riemann zeta function
- Selberg's zeta function and the spectral geometry of geometrically finite hyperbolic surfac
- Applications of Group Cohomology to the Classification of Fourier-Space Quasicrystals
- Twisted action of the symmetric group on the cohomology of a flag manifold, Banach Center P
- On the Cohomology of the Lie Algebra Arising from the Lower Central Series of a p-Group
- The Kazhdan property of the mapping class group of closed surfaces and the first cohomology
- WZW orientifolds and finite group cohomology
- Group cohomology of universal ordinary distribution
- Singularities of Prolonged Group Actions on Jet Bundles
- THE COMBINATORICS OF THE BAR RESOLUTION IN GROUP COHOMOLOGY

更多相关文章：
更多相关标签：