We study the asymptotic behavior of zeros of the Selberg zeta-function for the congruence subgroup Γ 0 (4) as a function of a one-parameter family of characters tending to the trivial character. The motivation for the study comes from observations based on numerical computations. Some of the observed phenomena lead to precise theorems that we prove and compare with the original numerical results.

Abstract. The standard realization of the Hecke algebra on classical holomorphic cusp forms and the corresponding period polynomials is well known. In this article we consider a nonstandard realization of the Hecke algebra on Maass cusp forms for the Hecke congruence subgroups Γ 0 (n). We show that the vector valued period functions derived recently by Hilgert, Mayer and Movasati as special eigenfunctions of the transfer operator for Γ 0 (n) are indeed related to the Maass cusp forms for these groups. This leads also to a simple interpretation of the "Hecke like" operators of these authors in terms of the aforementioned nonstandard realization of the Hecke algebra on the space of vector valued period functions.

The transfer operator for Γ 0 (N ) and trivial character χ 0 possesses a finite group of symmetries generated by permutation matrices P with P 2 = id. Every such symmetry leads to a factorization of the Selberg zeta function in terms of Fredholm determinants of a reduced transfer operator. These symmetries are related to the group of automorphisms in GL(2, Z) of the Maass wave forms of Γ 0 (N ) . For the group Γ 0 (4) and Selberg's character χ α there exists just one non-trivial symmetry operator P . The eigenfunctions of the corresponding reduced transfer operator with eigenvalue λ = ±1 are related to Maass forms even respectively odd under a corresponding automorphism. It then follows from a result of Sarnak and Phillips that the zeros of the Selberg function determined by the eigenvalues λ = −1 of the reduced transfer operator stay on the critical line under the deformation of the character. From numerical results we expect that on the other hand all the zeros corresponding to the eigenvalue λ = +1 leave this line for α turning away from zero.

the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

No abstract