Autonomous Sequence Generation for a Neural Dynamic Robot: Scene Perception, Serial Order, and Object-Oriented Movement

Abstract

Neurally inspired robotics already has a long history that includes reactive systems emulating reflexes, neural oscillators to generate movement patterns, and neural networks as trainable filters for high-dimensional sensory information. Neural inspiration has been less successful at the level of cognition. Decision-making, planning, building and using memories, for instance, are more often addressed in terms of computational algorithms than through neural process models. To move neural process models beyond reactive behavior toward cognition, the capacity to autonomously generate sequences of processing steps is critical. We review a potential solution to this problem that is based on strongly recurrent neural networks described as neural dynamic systems. Their stable states perform elementary motor or cognitive functions while coupled to sensory inputs. The state of the neural dynamics transitions to a new motor or cognitive function when a previously stable neural state becomes unstable. Only when a neural robotic system is capable of acting autonomously does it become a useful to a human user. We demonstrate how a neural dynamic architecture that supports autonomous sequence generation can engage in such interaction. A human user presents colored objects to the robot in a particular order, thus defining a serial order of color concepts. The user then exposes the system to a visual scene that contains the colored objects in a new spatial arrangement. The robot autonomously builds a scene representation by sequentially bringing objects into the attentional foreground. Scene memory updates if the scene changes. The robot performs visual search and then reaches for the objects in the instructed serial order. In doing so, the robot generalizes across time and space, is capable of waiting when an element is missing, and updates its action plans online when the scene changes. The entire flow of behavior emerges from a time-continuous neural dynamics without any controlling or supervisory algorithm.

A Developmental Framework for Embodiment Research: The Next Step Toward Integrating Concepts and Methods

“…For example, based on a computational approach to developmental systems neuroscience, Schöner et al (2018) simulated a simple neural dynamic model of movement generation, serving as platform for discussing infants' developmental challenges as they learn to reach for objects. Also, based on this model, they were able to construct a neural inspired robot imitating the neural processes underlying the reaching behavior (Tekülve et al, 2019). Although such modeling approaches are limited to engineering and computing capacities, they provide a potent tool to test hypotheses between different levels of embodiment, especially between the neural network level and the level of sensorimotor processing and motor actions.…”

Decoding Multiple Sound-Categories in the Auditory Cortex by Neural Networks: An fNIRS Study

“…RNNs are capable of predicting and classifying sequential data. They have been widely applied in robotics for various purposes and systems such as obstacle avoidance control (Xu et al, 2019b ; Zheng et al, 2019 ; Zhao et al, 2020 ), self-organizing robot control (Smith et al, 2020 ), collision-free compliance control (Zhou et al, 2019 ), dynamic neural robots (Tekulve et al, 2019 ), and self-driving system (Chen et al, 2019 ). Recently, RNNs have achieved impressive results in detecting seizures (Sirpal et al, 2019 ), brain injuries (Ieong et al, 2019 ), and pain (Hu et al, 2019b ), as well as in discriminating attention-deficit hyperactivity disorder (Dubreuil-Vall et al, 2020 ).…”

Optimum Integration Procedure for Connectionist and Dynamic Field Equations

“…Recent research indicates that connectionist and dynamic field theory (DFT) models might push developments in various branches of robotics (Schürmann et al, 2019;Tekülve et al, 2019;Schürmann and Beckerle, 2020;Torricelli et al, 2020) and specifically in embodied artificial cognitive systems (Lomp et al, 2016). Optimum integration procedures for these models may greatly contribute to the development of proper architectures, by accelerating simulation times, or equivalently, by reducing numerical errors.…”

“…To the best of our knowledge, in order to simulate connectionist and dynamic field models the Euler Method (EM) is widely used. It was used, for instance, in Thelen et al (2001) (as explicitly stated in page 21, second paragraph), in Verguts and Fias (2005) (according to a personal communication), in Lomp et al (2016), and in Tekülve et al (2019). In Lomp et al (2016), the authors explicitly stated that they use the EM and did not use higher-order numerical methods because they require very many function evaluations per time step, which defeats their computational advantage when each evaluation is computationally costly.…”