Intuitively, some pairs of neutron star models can be thought of as being 'closer' together than others, in the sense that more precise observations might be required to distinguish between them than would be necessary for other pairs. In this paper, borrowing ideas from the study of geometrodynamics, we introduce a mathematical formalism to define a geometric distance between stellar models, to provide a quantitative meaning for this notion of 'closeness'. In particular, it is known that the set of all Riemannian metrics on a manifold itself admits the structure of a Riemannian manifold ('configuration manifold'), which comes equipped with a canonical metric. By thinking of a stationary star as being a particular 3 + 1 metric, the structure of which is determined through the Tolman-Oppenheimer-Volkoff relations and their generalisations, points on a suitably restricted configuration manifold can be thought of as representing different stars, and distances between these points can be computed. We develop the necessary mathematical machinery to build the configuration manifold of neutron star models, and provide some worked examples to illustrate how this space might be used in future studies of neutron star structure.