Intrinsic Plasticity Based Inference Acceleration for Spiking Multi-Layer Perceptron

Zhang, Shuxun; Zhang, Anguo; Ma, Yupeng; Zhu, Wei

doi:10.1109/access.2019.2914424

Cited by 5 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In many ANN cases, bias is considered a common constant within a layer or even set to zero. We can expect significantly better performances by introducing intrinsic plasticity into ANN or spiking neurons (Zhang and Li, 2019 ; Zhang et al, 2019 ). A similarity between the simplified intrinsic plasticity introduced in ANN and batch normalization has also been reported (Shaw et al, 2020 ).…”

Section: Optimization Strategy: Multiscale Credit Assignmentmentioning

confidence: 99%

Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network

Jeon

Kim

2023

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Although it may appear infeasible and impractical, building artificial intelligence (AI) using a bottom-up approach based on the understanding of neuroscience is straightforward. The lack of a generalized governing principle for biological neural networks (BNNs) forces us to address this problem by converting piecemeal information on the diverse features of neurons, synapses, and neural circuits into AI. In this review, we described recent attempts to build a biologically plausible neural network by following neuroscientifically similar strategies of neural network optimization or by implanting the outcome of the optimization, such as the properties of single computational units and the characteristics of the network architecture. In addition, we proposed a formalism of the relationship between the set of objectives that neural networks attempt to achieve, and neural network classes categorized by how closely their architectural features resemble those of BNN. This formalism is expected to define the potential roles of top-down and bottom-up approaches for building a biologically plausible neural network and offer a map helping the navigation of the gap between neuroscience and AI engineering.

show abstract

Section: Optimization Strategy: Multiscale Credit Assignmentmentioning

confidence: 99%

Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network

Jeon

Kim

2023

Front. Comput. Neurosci.

View full text Add to dashboard Cite

show abstract

“…The output value is calculated as the input values passed through the layers sequentially in the forward direction, and the weights and biases in the model are trained through error back propagation [43]. MLP models are widely used for typical classification and regression tasks [44]. Neural network models, including MLP, have strong advantages, including high performance and no special processing requirements, but they have disadvantages, such as a large number of hyperparameters, proneness to overfitting, and a large computational burden.…”

Section: Neural Network Modelsmentioning

confidence: 99%

Motor Load Balancing with Roll Force Prediction for a Cold-Rolling Setup with Neural Networks

Lee

Son

2021

Mathematics

View full text Add to dashboard Cite

The use of machine learning algorithms to improve productivity and quality and to maximize efficiency in the steel industry has recently become a major trend. In this paper, we propose an algorithm that automates the setup in the cold-rolling process and maximizes productivity by predicting the roll forces and motor loads with multi-layer perceptron networks in addition to balancing the motor loads to increase production speed. The proposed method first constructs multilayer perceptron models with all available information from the components, the hot-rolling process, and the cold-rolling process. Then, the cold-rolling variables related to the normal part set-up are adjusted to balance the motor loads among the rolling stands. To validate the proposed method, we used a data set with 70,533 instances of 128 types of steels with 78 variables, extracted from the actual manufacturing process. The proposed method was found to be superior to the physical prediction model currently used for setups with regard to the prediction accuracy, motor load balancing, and production speed.

show abstract

“…In [53], an IP mechanism, depends on threshold level, was applied to a multi-layer SNN. The IP mechanism adapted the firing rates of neurons at a steady level while maximizing the information entropy.…”

Section: Related Workmentioning

confidence: 99%

Automated Adaptive Threshold-Based Feature Extraction and Learning for Spiking Neural Networks

Amin

2021

IEEE Access

View full text Add to dashboard Cite

Over the past years Spiking Neural Networks (SNNs) models became attractive as a possible bridge to enable low-power event-driven neuromorphic hardware. SNNs have a high computational power due to the implicit employment of the biologically inspired input times. SNNs employ various parameters such as neuron threshold, synaptic delays, and weights in their structures. However, SNNs applications are still limited and elementary compared with other neural network architectures such as the Convolution Neural Networks (CNNs). In this research, a new SNN-based model named Adaptive Threshold Module (ATM) and its algorithm are proposed. The proposed ATM and algorithm depend on the adaptation of the internal spiking neuron threshold level. Adapting the threshold of the neurons is employed to control the spiking neuron firing rate to uniquely extract the main features of the input pattern that is in the shape of spike trains. It is shown that this technique works as an automated feature extraction method of input patterns in an efficient and faster way than other methods. The proposed method can preserve all information of the input spike trains. Simulations of the proposed model and the algorithm, using the challenging speech TIDIGITS dataset, sound RWCP dataset, and Poisson distribution spike trains, show encouraging results. The ATM can make SNN provide an accuracy surpassing that of the current state-of-the-art SNN algorithms and conventional non-spiking learning models.

show abstract

Intrinsic Plasticity Based Inference Acceleration for Spiking Multi-Layer Perceptron

Cited by 5 publications

References 27 publications

Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network

Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network

Motor Load Balancing with Roll Force Prediction for a Cold-Rolling Setup with Neural Networks

Automated Adaptive Threshold-Based Feature Extraction and Learning for Spiking Neural Networks

Contact Info

Product

Resources

About