Let N be the space of Gaussian distribution functions over R, regarded as a 2-dimensional statistical manifold parameterized by the mean µ and the deviation σ. In this paper we show that the tangent bundle of N , endowed with its natural Kähler structure, is the Siegel-Jacobi space appearing in the context of Number Theory and Jacobi forms. Geometrical aspects of the Siegel-Jacobi space are discussed in detail (completeness, curvature, group of holomorphic isometries, space of Kähler functions, relationship to the Jacobi group), and are related to the quantum formalism in its geometrical form, i.e., based on the Kähler structure of the complex projective space.This paper is a continuation of our previous work [Mol12b, Mol12a, Mol13], where we studied the quantum formalism from a geometric and information-theoretical point of view.