Direct integral decompositions and multiplicities for induced representations of nilpotent Lie groups

Abstract

Let K K be a Lie subgroup of the connected, simply connected nilpotent Lie group G G , and let k \mathfrak {k} , g \mathfrak {g} be the corresponding Lie algebras. Suppose that σ \sigma is an irreducible unitary representation of K K . We give an explicit direct integral decomposition of Ind k → G σ {\operatorname {Ind} _{k \to G}}\sigma into irreducibles. The description uses the Kirillov orbit picture, which gives a bijection between G ∧ G^\wedge and the coadjoint orbits in g ∗ {\mathfrak {g}^{\ast }} (and similarly for K ∧ , k ∗ K^\wedge ,\,{\mathfrak {k}^{\ast }} ). Let P : k ∗ → g ∗ P:{\mathfrak {k}^{\ast }} \to {\mathfrak {g}^{\ast }} be the canonical projection, let O σ ⊂ k ∗ {\mathcal {O}_\sigma } \subset {\mathfrak {k}^{\ast }} be the orbit corresponding to σ \sigma , and, for π ∈ G ∧ \pi \in G^\wedge , let O π ⊂ g ∗ {\mathcal {O}_\pi } \subset {\mathfrak {g}^{\ast }} be the corresponding orbit. The main result of the paper says essentially that π ∈ G ∧ \pi \in G^\wedge appears in the direct integral iff P − 1 ( O σ ) {P^{ - 1}}({\mathcal {O}_\sigma }) meets O π {\mathcal {O}_\pi } ; the multiplicity of π \pi is the number of Ad ∗ ( K ) {\operatorname {Ad} ^{\ast }}(K) -orbits in O π ∩ P − 1 ( O σ ) {\mathcal {O}_\pi } \cap {P^{ - 1}}({\mathcal {O}_\sigma }) . There is also a natural description of the measure class in the integral.

Untitled

“…But it was the work of Pukanszky [18] and Vergne [21] that gave a good indication of how an orbital formula for an induced representation should look. Unfortunately, this was not really seized upon until recently in the work of Corwin and Greenleaf [2]. They give a formulation for the spectral decomposition in the nilpotent case purely in terms of orbital parameters.…”

“…This causes two problems-first, a very long and extremely complicated proof of the main formula; and second, the fact that their result is false for exponential solvable groups. Upon seeing [2], I felt that the proof could be simplified and that their formula-suitably altered-was valid for exponential solvable groups and perhaps more. This paper is the first in a series which will attempt to substantiate these feelings.…”

“…The first is to give proofs of these formulas for G nilpotent. As mentioned earlier the proofs in [2,4] (of somewhat different formulas) rely totally on "layers," are long and complex, and are completely tied to the nilpotent situation. Ours will be by induction on dim G/H and have the capability of generalization.…”

“…Part I begins in §0 with a statement of the basic results on induced and restricted representations when H is codimension 1 in a nilpotent G. (These have been well known since [9].) In §1 we reformulate the main result of [2]. The reformulated result-Theorem 1.5-is formula (I) for the case that v is a character.…”

“…The precise result is in Theorem 4.2. The last section in Part I ( §5) gives a new proof of further refinements of the multiplicity function from [2]. Using some real algebraic geometry, it is proved that the multiplicity is either uniformly infinite or bounded finite.…”