Space Emerges from What We Know—Spatial Categorisations Induced by Information Constraints

Abstract

Seeking goals carried out by agents with a level of competency requires an “understanding” of the structure of their world. While abstract formal descriptions of a world structure in terms of geometric axioms can be formulated in principle, it is not likely that this is the representation that is actually employed by biological organisms or that should be used by biologically plausible models. Instead, we operate by the assumption that biological organisms are constrained in their information processing capacities, which in the past has led to a number of insightful hypotheses and models for biologically plausible behaviour generation. Here we use this approach to study various types of spatial categorizations that emerge through such informational constraints imposed on embodied agents. We will see that geometrically-rich spatial representations emerge when agents employ a trade-off between the minimisation of the Shannon information used to describe locations within the environment and the reduction of the location error generated by the resulting approximate spatial description. In addition, agents do not always need to construct these representations from the ground up, but they can obtain them by refining less precise spatial descriptions constructed previously. Importantly, we find that these can be optimal at both steps of refinement, as guaranteed by the successive refinement principle from information theory. Finally, clusters induced by these spatial representations via the information bottleneck method are able to reflect the environment’s topology without relying on an explicit geometric description of the environment’s structure. Our findings suggest that the fundamental geometric notions possessed by natural agents do not need to be part of their a priori knowledge but could emerge as a byproduct of the pressure to process information parsimoniously.

Exact and Soft Successive Refinement of the Information Bottleneck

“…As our numerical experiments suggest that the IB problem is not always successively refinable, it is desirable to quantify the lack of successive refinement—i.e., the lack of informational optimality induced by several-stage processing. These considerations lead to the notion of soft successive refinement [ 18 ], which we define and motivate in this section. As we will see, this generalisation of exact SR does not depend on the specific structure of the IB setting; rather, it can also be used as a generalisation of exact SR for any rate-distortion scenario.…”

“…For instance, choosing X to be an agent’s past and Y to be its future leads it to extract the most relevant features of its environment [ 3 , 4 , 5 , 6 ]. More generally, the IB point of view on modelling embodied agents’ representations has been leveraged for unifying efficient and predictive coding principles in theoretical neuroscience—at the level of single neurons [ 3 , 7 , 8 , 9 ] and neuronal populations [ 9 , 10 , 11 , 12 , 13 ]—but also for studying sensor evolution [ 14 , 15 , 16 ], the emergence of common concepts [ 17 ] and of spatial categories [ 18 ], the evolution of human language [ 19 , 20 , 21 ], or for implementing informationally efficient control in artificial agents [ 22 , 23 , 24 ]. This line of research brings increasing support to the hypothesis that, particularly for evolutionary reasons, biological agents are often poised close to optimality in the IB sense.…”

“…Second, we soften [ 18 ] the traditional notion of successive refinement and study the extent to which several-stage processing incurs a loss of information optimality. More precisely, we propose to measure soft successive refinement with the unique information [ 40 ] (UI) that the coarser bottleneck holds about the source X , as compared to the finer one .…”

“…The loss of information optimality induced by several-stage processing has already been studied in [ 60 ] (see next paragraph), but a quantification of it based on soft Markovianity was, to the best of our knowledge, only considered in [ 18 ]. Here, we take inspiration in the latter work to quantify soft successive refinement, but we explicitly address the problem that joint distributions over distinct bottlenecks are not uniquely defined.…”

A space of goals: the cognitive geometry of informationally bounded agents

“…The cost for the cognitive processing required to operate a given policy has also been measured in terms of information-theoretic functionals [12][13][14][15]. In the present paper, we are interested in the geometries emerging from the incorporation of such informational cognitive costs [16].…”