Behavior control in the sensorimotor loop with short-term synaptic dynamics induced by self-regulating neurons

Abstract

The behavior and skills of living systems depend on the distributed control provided by specialized and highly recurrent neural networks. Learning and memory in these systems is mediated by a set of adaptation mechanisms, known collectively as neuronal plasticity. Translating principles of recurrent neural control and plasticity to artificial agents has seen major strides, but is usually hampered by the complex interactions between the agent's body and its environment. One of the important standing issues is for the agent to support multiple stable states of behavior, so that its behavioral repertoire matches the requirements imposed by these interactions. The agent also must have the capacity to switch between these states in time scales that are comparable to those by which sensory stimulation varies. Achieving this requires a mechanism of short-term memory that allows the neurocontroller to keep track of the recent history of its input, which finds its biological counterpart in short-term synaptic plasticity. This issue is approached here by deriving synaptic dynamics in recurrent neural networks. Neurons are introduced as self-regulating units with a rich repertoire of dynamics. They exhibit homeostatic properties for certain parameter domains, which result in a set of stable states and the required short-term memory. They can also operate as oscillators, which allow them to surpass the level of activity imposed by their homeostatic operation conditions. Neural systems endowed with the derived synaptic dynamics can be utilized for the neural behavior control of autonomous mobile agents. The resulting behavior depends also on the underlying network structure, which is either engineered or developed by evolutionary techniques. The effectiveness of these self-regulating units is demonstrated by controlling locomotion of a hexapod with 18 degrees of freedom, and obstacle-avoidance of a wheel-driven robot.

Closed-loop Robots Driven by Short-Term Synaptic Plasticity: Emergent Explorative vs. Limit-Cycle Locomotion

“…STSP has been argued, besides others, to be relevant or causal for working memory (Barak and Tsodyks, 2014 ), for the facilitation of time sequences of alternating neural populations (Carrillo-Reid et al, 2015 ), for motor control in general (Nadim and Manor, 2000 ), and for the sculpting of rhythmic motor patterns (Jia and Parker, 2016 ) in particular. Plasticity mechanisms similar to STSP have also been shown to allow for stable gaits (Toutounji and Pasemann, 2014 ) in neural networks which are distinctively simpler than the ones used conventionally for bio-inspired controllers (Schilling et al, 2013 ).…”

Synaptic plasticity in a recurrent neural network for versatile and adaptive behaviors of a walking robot

“…Note that turning angle or turning duration is basically derived from the width of the hysteresis. According to this, Toutounji and Pasemann ( 2014 ) introduced short-term plasticity induced by self-regulating neurons (Zahedi and Pasemann, 2007 ) in MRC. This allows the wheel-driven robot ALICE with five distance sensors to capable of avoiding sharp corners.…”

“…which can be exploited for signal processing and locomotion generation (Steingrube et al, 2010 ; von Twickel et al, 2011 ; Toutounji and Pasemann, 2014 ). According to this, many studies mainly employ ANNs for the purpose of locomotion (Beer et al, 1997 ; Valsalam and Miikkulainen, 2008 ; Lewinger and Quinn, 2011 ; von Twickel et al, 2011 ; Von Twickel et al, 2012 ; Schilling et al, 2013 ; Toutounji and Pasemann, 2014 ). For example, Beer et al ( 1997 ) developed a distributed neural network controller of a six-legged walking machine for generating locomotion with reflex actions to deal with irregular, slatted, and compliant terrains.…”

“…Only a few works used ANNs for walking and navigating in simple cluttered environments with a few obstacles (Filliat et al, 1999 ; Fischer et al, 2004 ). Usually, most developed neural mechanisms (Filliat et al, 1999 ; Fischer et al, 2004 ; Von Twickel et al, 2012 ; Toutounji and Pasemann, 2014 ) use evolutionary techniques to optimize neural parameters (i.e., synaptic weights and structures). Thus, they typically end up with complex massive recurrent connectivity structures which is difficult to understand and be mathematically analyzed.…”

Homeostatic Plasticity for Single Node Delay-Coupled Reservoir Computing

“…The choice of the operating regime depends on the functionality that a model of homeostatic plasticity aims to achieve. This resulted in many flavors of homeostatic plasticity for regulating recurrent neural networks in computational neuroscience (Somers, Nelson, & Sur, 1995;Soto-Treviño, Thoroughman, Marder, & Abbott, 2001;Renart, Song, & Wang, 2003;Lazar et al, 2007Lazar et al, , 2009Marković and Gros, 2012;Remme & Wadman, 2012;Naudé, Cessac, Berry, & Delord, 2013;Zheng, Dimitrakakis, & Triesch, 2013;Toutounji & Pipa, 2014), neurorobotics (Williams & Noble, 2007;Vargas, Moioli, Von Zuben, & Husbands, 2009;Hoinville, Siles, & Hénaff, 2011;Dasgupta, Wörgötter, & Manoonpong, 2013;Toutounji & Pasemann, 2014), and reservoir computing (Schrauwen, Wardermann, Verstraeten, Steil, & Stroobandt, 2008;Dasgupta, Wörgötter, & Manoonpong, 2013). Here, we use a homeostatic plasticity mechanism to regulate the v-delays so as to balance responsiveness to the input and its history on the one hand, against optimal expansion of its informational features into the high-dimensional phase space of the system, on the other hand.…”