Where Do I Move My Sensors? Emergence of a Topological Representation of Sensors Poses From the Sensorimotor Flow

Abstract

This paper deals with the perception of mobile robotic systems within the framework of interactive perception, and inspired by the sensorimotor contingencies (SMC) theory. These approaches state that perception arises from active exploration of an environment. In the SMC theory, it is postulated that information about the structure of space could be recovered from a quasi-uninterpreted sensorimotor flow. In a recent article, the authors have provided a mathematical framework for the construction of a sensorimotor representation of the interaction between the sensors and the body of a naive agent, provided that the sensory inputs come from the agent's own body. An extension of these results, with stimulations coming from an unknown changing environment, is proposed in this paper. More precisely it is demonstrated that, through repeated explorations of its motor configurations, the perceived sensory invariants can be exploited to build a topologically accurate internal representation of the relative poses of the agent's sensors in the physical world. Precise theoretical considerations are provided as well as an experimental framework. Finally, some examples that serve as proofs of concepts are analysed in both simulated and realistic environments.

A Formal Account of Structuring Motor Actions With Sensory Prediction for a Naive Agent

“…To begin, let us consider the case where m 1 and m 2 are fixed, so that the agent is only able to move its arm supporting a camera-like sensor generating a sensation s, i.e., m is restricted to (m 3 , m 4 , m 5 ) only. In such a scenario, one has a fixed base agent for which each motor configuration m can be mapped to one corresponding sensor pose, which is itself mapped to a sensation s. This simple statement allows to build structures in M by exploiting only the sensorimotor flow (m, s), structures that can be leveraged to build an internal representation of the agent body; they can be further refined into a representation of its peripersonal space (Marcel et al, 2017;Marcel et al, 2019). In these works, m carries all spatial data, possibly with some redundancy, about the coupling between the agent and its environment: the combination of states m and e is sufficient to determine the resulting sensory output s (as described by the formal sensorimotor map ψ).…”

“…In the authors' previous works (Marcel et al, 2017 ; Marcel et al, 2019 ), sensorimotor interaction occurred as a sequence of (generally discrete) steps where at each point, the agent could access both its proprioception (seen as an array of current joint configuration states) and its corresponding exteroceptive array s = ψ( m , ϵ ). These sensory arrays were then compared for equality (and for equality only ) as total vectors , that is the agent may not access the vectors component by component.…”

“…This article follows much of the same approach begun in Philipona et al (2003) and subsequently developed in e.g., Marcel et al (2017), Marcel et al (2019), Laflaquière et al (2018), and Laflaquière and Ortiz (2019). In particular, it sets out to mathematically describe from an exterior, "objective, " point of view some properties of the interaction between the robot and its environment which should appear in its sensorimotor flow.…”

“…Early works on SMCT have been shown to lead to discovery of the color spectrum (Philipona and O'Regan, 2006 ) and of the dimensionality of ambient space (Philipona et al, 2003 , 2004 ; Laflaquière et al, 2010 ; Laflaquière et al, 2012 ) in “naive” agents, which can be extended to that of an internal, path independent, notion of space (Terekhov and O'Regan, 2016 ). We already proposed different contributions in this field, successively dealing with peripersonal space characterization (Laflaquière et al, 2015 ), self-contact and body representation (Marcel et al, 2017 ) and the emergence of a topological representation of sensors poses (Marcel et al, 2019 ). These works, as well as Laflaquière et al ( 2018 ) and Laflaquière and Ortiz ( 2019 ), devote a significant effort to providing formalisms suited to make explicit, and, where applicable, formally prove—not only the processing required to capture the contingencies, i.e., invariants in the sensorimotor flow of the agent, but also the mechanisms by which said contingencies should appear.…”

“…In accordance with the previous remark about genericity, a particular attention is given to the construction of said formalism with assumptions and proofs explicitly detailed. Moreover, the bootstrapping aspect is emphasized throughout the work, much in the spirit of Marcel et al ( 2017 ); Marcel et al ( 2019 ), highlighting the distinction between the points of view of the agent and of the observer in the description of the problems and an explicit discussion of the degree of a priori knowledge given to the agent, in terms of both data and computations available to it. There lie two contributions of this article: while the formalism is used to formally describe why and how spatial coherence lead to the discovery of sensory prediction very much like alluded to in Laflaquière ( 2017 ) and how this sensory prediction encodes spatial structure akin to that of Terekhov and O'Regan ( 2016 ) and Laflaquière and Ortiz ( 2019 ), it does so with a greater emphasis put on the precise relations between the algebraic structures at play and with much weakened assumptions about a priori capabilities of the agent, much closer to those put forward in O'Regan and Noë ( 2001 ).…”