Novel plasticity rule can explain the development of sensorimotor intelligence

Abstract

Grounding autonomous behavior in the nervous system is a fundamental challenge for neuroscience. In particular, self-organized behavioral development provides more questions than answers. Are there special functional units for curiosity, motivation, and creativity? This paper argues that these features can be grounded in synaptic plasticity itself, without requiring any higher-level constructs. We propose differential extrinsic plasticity (DEP) as a new synaptic rule for self-learning systems and apply it to a number of complex robotic systems as a test case. Without specifying any purpose or goal, seemingly purposeful and adaptive rhythmic behavior is developed, displaying a certain level of sensorimotor intelligence. These surprising results require no systemspecific modifications of the DEP rule. They rather arise from the underlying mechanism of spontaneous symmetry breaking, which is due to the tight brain body environment coupling. The new synaptic rule is biologically plausible and would be an interesting target for neurobiological investigation. We also argue that this neuronal mechanism may have been a catalyst in natural evolution.R esearch in neuroscience produces an understanding of the brain on many different levels. At the smallest scale, there is enormous progress in understanding mechanisms of neural signal transmission and processing (1-4). At the other end, neuroimaging and related techniques enable the creation of a global understanding of the brain's functional organization (5, 6). However, a gap remains in binding these results together, which leaves open the question of how all these complex mechanisms interact (7-9). This paper advocates for the role of self-organization in bridging this gap. We focus on the functionality of neural circuits acquired during individual development by processes of self-organizationmaking complex global behavior emerge from simple local rules. Donald Hebb's formula "cells that fire together wire together" (10) may be seen as an early example of such a simple local rule which has proven successful in building associative memories and perceptual functions (11,12). However, Hebb's law and its successors like BCM (13) and STDP (14, 15) are restricted to scenarios where the learning is driven passively by an externally generated data stream. However, from the perspective of an autonomous agent, sensory input is mainly determined by its own actions. The challenge of behavioral self-organization requires a new kind of learning that bootstraps novel behavior out of the self-generated past experiences. This paper introduces a rule which may be expressed as "chaining together what changes together." This rule takes into account temporal structure and establishes contact to the external world by directly relating the behavioral level to the synaptic dynamics. These features together provide a mechanism for bootstrapping behavioral patterns from scratch.This synaptic mechanism is neurobiologically plausible and raises the question of whether it is present in living beings...

In Search for the Neural Mechanisms of Individual Development: Behavior-Driven Differential Hebbian Learning

“…This is reminiscent of differential Hebbian learning studied in earlier work, see Kosko (1986), Klopf (1988), Roberts (1999), and Lowe et al (2011). The advantage of differential over pure Hebbian learning for the self-organized behavior acquisition has been discussed in a concrete setting close to that of this paper in Der and Martius (2015). On the other hand, different from any Hebbian-like learning, the postsynaptic rate is not that of the neuron itself but is generated by a feedback chain from the external world the neuron is controlling.…”

Quantifying Emergent Behavior of Autonomous Robots

“…For m = 3, the value of c MT is 0.23 for 0.1 < ε < 0.8 because of the shallow region at [0. 8,9]. This leads to the value of 0.68 (m * c MT m ) for the state complexity in Fig 3(b).…”

“…This also applies in principle to behavior from task-independent 1 objectives that have been recently more and more successful in generating emergent autonomous behavior in robots [1,14,24,32,35,39,48]. However, there are several cases of emergent behavior where this strategy fails: if the behavior arises from a) optimizing a local function [7,28], b) optimizing a crude approximation of a computationally expensive objective function [28], c) local interaction rules without an explicit optimization function [8,9,37], and d) a biological system (e. g. freely moving animals) where we don't know the underlying optimization function. Thus, independent of the origin of behavior it would be useful to have a quantitative description of its structure.…”

Self-Organized Behavior Generation for Musculoskeletal Robots

“…In Der and Martius (2015), the emergence of rotational modes was demonstrated for a humanoid robot with revolution joints and in simulation. With the MyoArm, we have a much more challenging situation.…”

“…The above learning rule differs from the DEP rule introduced in Der and Martius (2015) by the normalization factor ||ẋ|| −2 introduced with Equation (6) above. In the experiments this leads to a more continuous activity in the behaviors avoiding potential pauses of inactivity.…”