Active learning of local predictable representations with artificial curiosity

Abstract

In this article, we present some preliminary work on integrating an artificial curiosity mechanism in PROPRE, a generic and modular neural architecture, to obtain online, openended and active learning of a sensory-motor space, where large areas can be unlearnable. PROPRE consists of the combination of the projection of the input motor flow, using a self-organizing map, with the regression of the sensory output flow from this projection representation, using a linear regression. The main feature of PROPRE is the use of a predictability module that provides an interestingness measure for the current motor stimulus depending on a simple evaluation of the sensory prediction quality. This measure modulates the projection learning so that to favor the representations that predict the output better than a local average. Especially, this leads to the learning of local representations where an input/output relationship is defined [1]. In this article, we propose an artificial curiosity mechanism based on the monitoring of learning progress, as proposed in [2], in the neighborhood of each local representation. Thus, PROPRE simultaneously learns interesting representations of the input flow (depending on their capacities to predict the output) and explores actively this input space where the learning progress is the higher. We illustrate our architecture on the learning of a direct model of an arm whose hand can only be perceived in a restricted visual space. The modulation of the projection learning leads to a better performance and the use of the curiosity mechanism provides quicker learning and even improves the final performance.

Curiosity‐based learning in infants: a neurocomputational approach

“…For example, curiosity could be underpinned by a drive to maximize learning progress by interacting with the environment in a novel manner relative to previously encountered events (Oudeyer et al., ). Alternatively, curiosity could be driven by prediction mechanisms, allowing the system to engage in activities for which predictability is maximal (Lefort & Gepperth, ) or minimal (Botvinick, Niv, & Barto, ). Still other approaches assume that curiosity involves maximizing a system's competence or ability to perform a task (Murakami, Kroger, Birkholz, & Triesch, ).…”

A Generative Learning Approach to Sensor Fusion and Change Detection

“…1, to address the proposed fusion problem. It is conceptually similar to several of our previous works in multi-sensory fusion [17], incremental learning [18][19][20] and developmental learning [21,22]. In the light of the evaluation criteria for multi-sensory fusion approaches proposed in Sec.…”

Lessons from infant learning for unsupervised machine learning

“…Over the years, different types of curiosity mechanism have been proposed for artificial systems. Some researchers suggest that curiosity could be prediction-based, causing agents to attend to input for which predictability is minimal 192 or maximal 193 . In the context of curriculum learning, ref.…”