Adaptive Thresholds for Neural Networks With Synaptic Noise

Abstract

The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of both layered feedforward and fully connected neural network models with synaptic noise. These two types of architectures require a different method to be solved numerically. In both cases it is shown that, if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed. This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.

Adaptive threshold in TiO2-based synapses

“…On the other side, when the synaptic connection is not so strong (high resistance value), a punishment is produced increasing the voltage threshold for the resistive switching. It is widely known that different synapses differ in their ability to regulate or influence the activation of a biochemical mechanism (Ca 2+ concentration) for synaptic weight changes [30]. How the thresholds are modified by the previous history of reinforcing feedback resembles the change in the dynamics of Ca 2+ signals and is not reproduced by classical STDP implementations, being needed an adaptive threshold behavior as the one shown by our Ni/TiO 2 /Ni/Au devices.…”

“…This behavior is similar to the adaptive threshold mechanism of biological synapses. In the neural network context, the adaptive threshold can favor (reward) synaptic connection when the voltage threshold for the resistive switching is decreased due to a low resistance value of the corresponding synapse [30]. On the other side, when the synaptic connection is not so strong (high resistance value), a punishment is produced increasing the voltage threshold for the resistive switching.…”

Predicting the Earthquake Magnitude Using the Multilayer Perceptron Neural Network with Two Hidden Layers

“…Neural networks have been used successfully to solve complicated pattern recognition and classification problems in different domains such as image and object recognition [3], optimization and nonlinear programming [4], construction engineering [5] and [6], video and audio analysis [7], and financial forecasting [8].…”

On the Biological Plausibility of Artificial Metaplasticity

“…Understanding metaplasticity may yield new insights into how the modification of synapses is regulated and how information is stored by synapses in the brain. [7,8,9] This paper is organized as follows. Section 2 provides a brief introduction to related concepts (e.g.…”