It may be time to improve the neuron of artificial neural network

Abstract

<p>Artificial neural networks (ANNs) have won numerous contests in pattern recognition, machine learning, or artificial intelligence in recent years. The neuron of ANNs was designed by the stereotypical knowledge of biological neurons 70 years ago. Artificial Neuron is expressed as f(wx+b) or f(WX). This design does not consider dendrites' information processing capacity. However, some recent studies show that biological dendrites participate in the pre-calculation of input data. Concretely, biological dendrites play a role in extracting the interaction information among inputs (features). Therefore, it may be time to improve the neuron of ANNs. According to our previous studies (DD), this paper adds the dendrites' function to artificial Neuron. The dendrite function can be expressed as W<sup>i,i-1</sup>A<sup>i-1</sup> ○ A<sup>0|1|2|...|i-1</sup> . The generalized new neuron can be expressed as f(W(W<sup>i,i-1</sup>A<sup>i-1</sup> ○ A<sup>0|1|2|...|i-1</sup>)).The simplified new neuron be expressed as f(∑(WA ○ X)). After improving the neuron, there are so many networks to try. This paper shows some basic architecture for reference in the future. </p> <p>Interesting things: (1) The computational complexity of dendrite modules (W<sup>i,i-1</sup>A<sup>i-1</sup> ○ A<sup>i-1</sup>) connected in series is far lower than Horner's method. Will this speed up the calculation of basic functions in computers? (2) The range of sight of animals has a gradient, but the convolution layer does not have this characteristic. This paper proposes receptive fields with a gradient. (3) The networks using Gang neurons can delete traditional networks' Fully-connected Layer. In other words, the Fully-connected Layers' parameters are assigned to a single neuron, which reduces the parameters of a network for the same mapping capacity. </p><p>One important thing: ResDD can replace the current all ANNs' Neurons (ResDD modules+One Linear module)! ResDD has controllable precision for better generalization capability! </p><p>Gang neuron code is available at https://github.com/liugang1234567/Gang-neuron.</p>

Dynamic corticomuscular multi-regional modulations during finger movement revealed by time-varying network analysis

“…The transfer learning between the two signal modalities can be successfully carried out through the convolutional neural networks [20]. When using both EEG and electromyogram (EMG) as input of an artificial neural network [21], the accuracy of classification increases significantly [22]. The corticomuscular network has small-world characteristics, which will contribute to our understanding of human movements and facilitate movement recognition [7].…”