Adaptative time constants improve the prediction capability of recurrent neural networks

Draye, Jean-Philippe; Pavisic, Davor; Chéron, Guy; Libert, G.

doi:10.1007/bf02311573

Cited by 11 publications

(10 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Successful trainings were also used to produce kinematic patterns from unknown inputs (Figure 1B) aiming to produce walking for multiple purposes, such as virtual avatars or robotic exoskeletons. The training is supervised, involving learning rule adaptations of synaptic weights and time constant of each unit (Draye et al, 1995, 1996). A specific training procedure using Almeida algorithm was used to optimize learning performance (Cheron et al, 2011).…”

Section: Methodsmentioning

confidence: 99%

“…Introduction of timing allows modeling of more complex frequency behavior, improves the non-linearity effect of the sigmoid function and the memory effect of time delays (Draye et al, 1995). The distribution of the time constant and the synaptic weights between units (Draye et al, 1996) after learning was analyzed after multiple pattern learning and prediction.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Biological oscillations for learning walking coordination: dynamic recurrent neural network functionally models physiological central pattern generator

Hoellinger

Petieau

Duvinage

et al. 2013

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

The existence of dedicated neuronal modules such as those organized in the cerebral cortex, thalamus, basal ganglia, cerebellum, or spinal cord raises the question of how these functional modules are coordinated for appropriate motor behavior. Study of human locomotion offers an interesting field for addressing this central question. The coordination of the elevation of the 3 leg segments under a planar covariation rule (Borghese et al., 1996) was recently modeled (Barliya et al., 2009) by phase-adjusted simple oscillators shedding new light on the understanding of the central pattern generator (CPG) processing relevant oscillation signals. We describe the use of a dynamic recurrent neural network (DRNN) mimicking the natural oscillatory behavior of human locomotion for reproducing the planar covariation rule in both legs at different walking speeds. Neural network learning was based on sinusoid signals integrating frequency and amplitude features of the first three harmonics of the sagittal elevation angles of the thigh, shank, and foot of each lower limb. We verified the biological plausibility of the neural networks. Best results were obtained with oscillations extracted from the first three harmonics in comparison to oscillations outside the harmonic frequency peaks. Physiological replication steadily increased with the number of neuronal units from 1 to 80, where similarity index reached 0.99. Analysis of synaptic weighting showed that the proportion of inhibitory connections consistently increased with the number of neuronal units in the DRNN. This emerging property in the artificial neural networks resonates with recent advances in neurophysiology of inhibitory neurons that are involved in central nervous system oscillatory activities. The main message of this study is that this type of DRNN may offer a useful model of physiological central pattern generator for gaining insights in basic research and developing clinical applications.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Biological oscillations for learning walking coordination: dynamic recurrent neural network functionally models physiological central pattern generator

Hoellinger

Petieau

Duvinage

et al. 2013

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…They are thus particularly suitable for adaptive temporal processing (e.g., system identification, time series prediction, and control). In previous work, we proved that the recurrent aspect of the model greatly improves the network performance (Draye et al 1995.…”

Section: Identification Of the Emg-motion Relationship Using Dynamic mentioning

confidence: 99%

Self-selected modular recurrent neural networks with postural and inertial subnetworks applied to complex movements

Draye¹,

Winters

Chéron³

2002

Biological Cybernetics

Self Cite

View full text Add to dashboard Cite

It has been shown that dynamic recurrent neural networks are successful in identifying the complex mapping relationship between full-wave-rectified electromyographic (EMG) signals and limb trajectories during complex movements. These connectionist models include two types of adaptive parameters: the interconnection weights between the units and the time constants associated to each neuron-like unit; they are governed by continuous-time equations. Due to their internal structure, these models are particularly appropriate to solve dynamical tasks (with time-varying input and output signals). We show in this paper that the introduction of a modular organization dedicated to different aspects of the dynamical mapping including privileged communication channels can refine the architecture of these recurrent networks. We first divide the initial individual network into two communicating subnetworks. These two modules receive the same EMG signals as input but are involved in different identification tasks related to position and acceleration. We then show that the introduction of an artificial distance in the model (using a Gaussian modulation factor of weights) induces a reduced modular architecture based on a self-elimination of null synaptic weights. Moreover, this self-selected reduced model based on two subnetworks performs the identification task better than the original single network while using fewer free parameters (better learning curve and better identification quality). We also show that this modular network exhibits several features that can be considered as biologically plausible after the learning process: self-selection of a specific inhibitory communicating path between both subnetworks after the learning process, appearance of tonic and phasic neurons, and coherent distribution of the values of the time constants within each subnetwork.

show abstract

“…The time constants will be considered in the learning process too because they play an important role in the dynamical performance of the network (Draye et al, 1995;1996). the activation function, Xj the total input of the neuron, the W kj the weights for the k incoming signals Yk on neuron i and OJ an external input or bias.…”

Section: Time Dependent Recurrent Backpropagation: Learning Rulesmentioning

confidence: 99%

Plausible Neural Networks for Biological Modelling

Mastebroek¹,

Vos²

2001

Mathematical Modelling: Theory and Applications

View full text Add to dashboard Cite

This series is aimed at publishing work dealing with the definition, development and application of fundamental theory and methodology, computational and algorithmic implementations and comprehensive empirical studies in mathematical modelling. Work on new mathematics inspired by the construction of mathematical models, combining theory and experiment and furthering the understanding of the systems being modelled are particularly welcomed.Manuscripts to be considered for publication lie within the following, non-exhaustive list of areas: mathematical modelling in engineering, industrial mathematics, control theory, operations research, decision theory, economic modelling, mathematical programmering, mathematical system theory, geophysical sciences, climate modelling, environmental processes, mathematical modelling in psychology, political science, sociology and behavioural sciences, mathematical biology, mathematical ecology, image processing, computer vision, artificial intelligence, fuzzy systems, and approximate reasoning, genetic algorithms, neural networks, expert systems, pattern recognition, clustering, chaos and fractals.Original monographs, comprehensive surveys as well as edited collections will be considered for publication.Preface 5 times between neurons are introduced, clusters of high-value weights among the neurons in the integrator emerge during the training stage. It is concluded that fixed sign synapses and time delays in the information signalling process between neurons induce the creation of iterated patterns in the neural integrator. Chapter 6: Pattern Segmentation in an Associative Network of Spiking Neurons. The segmentation of a complex stimulus like a visual scene into a set of coherent patterns (objects) is an aspect of perception that underlies tasks like image processing, figure-ground segregation and object recognition. In the correlation theory of Malsburg (1981) an object is represented by the temporal correlation of the firing activities of (spatially) scattered cells. Multiple objects are represented by different firing patterns. Ritz uses a model of an associative network of spiking neurons to demonstrate the power of this theory. A realworld application and the current experimental evidence in support of this hypothesis are discussed. Chapter 7: Cortical Models for Movement Control. In this chapter Bullock describes the use of precise neuro-anatomical and neurophysiological facts in his model of voluntary motor control. Structures in areas 4 and 5 of the cortex can be identified with functional clusters of his VITE model. Later, sensory input is added to the model, and also the projections from the basal ganglia and the cerebellar cortex. The model is sufficiently accurate for making statements on the origin of some pathological conditions. In addition it is able to simulate the complicated activity of cursive handwriting. Chapter 8: Implications of Activity Dependent Processes in Spinal Cord Circuits for the Development of Motor Control. In their neural network model study Van...

show abstract

Adaptative time constants improve the prediction capability of recurrent neural networks

Cited by 11 publications

References 12 publications

Biological oscillations for learning walking coordination: dynamic recurrent neural network functionally models physiological central pattern generator

Biological oscillations for learning walking coordination: dynamic recurrent neural network functionally models physiological central pattern generator

Self-selected modular recurrent neural networks with postural and inertial subnetworks applied to complex movements

Plausible Neural Networks for Biological Modelling

Contact Info

Product

Resources

About