On Exploiting Haptic Cues for Self-Supervised Learning of Depth-Based Robot Navigation Affordances

Abstract

This article presents a method for online learning of robot navigation a↵or-dances from spatiotemporally correlated haptic and depth cues. The method allows the robot to incrementally learn which objects present in the environment are actually traversable. This is a critical requirement for any wheeled robot performing in natural environments, in which the inability to discern vegetation from non-traversable obstacles frequently hampers terrain progression. A wheeled robot prototype was developed in order to experimentally validate the proposed method. The robot prototype obtains haptic and depth sensory feedback from a pan-tilt telescopic antenna and from a structured light sensor, respectively. With the presented method, the robot learns a mapping between objects' descriptors, given the range data provided by the sensor, and objects' sti↵ness, as estimated from the interaction between the antenna and the object. Learning confidence estimation is considered in order to progressively reduce the number of required physical interactions with acquainted objects. To raise the number of meaningful interactions per object under time pressure, the several segments of the object under analysis are prioritised according to a set of morphological criteria. Field trials show the ability of the robot to progressively learn which elements of the environment are traversable.

Self-Supervised Learning of Motion-Induced Acoustic Noise Awareness in Social Robots

“…Self-supervised learning has been attracting considerable attention, in particular in safe navigation domain, which requires the robot to autonomously learn classi�iers for terrain assessment from images and point clouds [3,4,10,22,23,29,30]. In general, in this previous work the robot is asked to learn what perceptual features better predict a given robot-terrain interaction, provided ground-truth labels produced by an active perception process.…”

“…In general, in this previous work the robot is asked to learn what perceptual features better predict a given robot-terrain interaction, provided ground-truth labels produced by an active perception process. For instance, by manipulating an object, the robot is able to obtain ground-truth regarding how traversable that object is [4]. The learned associative mapping can then be used to predict future robot-terrain interactions, given sensory feedback.…”

Persistent self-supervised learning: From stereo to monocular vision for obstacle avoidance

“…Since the complementary cue then has to persist in the absence of the original cue, this form of SSL can be termed “persistent SSL.” Baleia et al. 11 study a rover with a haptic antenna sensor. In their application of terrain mapping, they try to map monocular cues to obstacles based on earlier events of encountering similar situations that resulted in either a hard obstacle, a traversable obstacle, or a clear path.…”

“…More recent studies aim to replace the function of the original cue with that of the complementary cue. [11][12][13][14] For instance, in Baleia et al, 11 the sense of touch is used to teach a vision process how to recognize traversable paths through vegetation with the goal of gradually reducing time-intensive haptic interaction. Hence, the learning of recognizing the complementary cue will have to persist in time.…”

Autonomous object modeling and exploiting: A new approach based on affordances from continual interaction with environment

“…Point 1) is well studied in literature. For example, Pfeifer and Scheier [12], Ugur et al [16] and Baleia [1] proposed approaches where an agent learns to define and classify objects of its environment through possibilities of interaction (such as "walkthroughable" or not). However, these approaches do not let the possibility to generate spatial behaviors.…”