While large-scale recording techniques indicate that the activity of heterogeneous neuronal populations lies on low-dimensional “neural manifolds”, it has remained challenging to reconcile this picture with the classical view of precisely tuned neurons interacting with each other in some ordered circuit structure. Using a modelling approach, we provide a conceptual, yet mathematically precise, link between these two contrasting views. We first show that there is no unique relationship between the circuit structure and the emergent low-dimensional dynamics that characterise the population activity. We then propose a method for retrieving the circuit structure from recordings of the population activity and test it on artificial data. Our approach provides not only a unifying framework for circuit and field models on one side, and low-rank networks on the other side, but also opens the perspective to identify principles of circuit structure from large-scale recordings.