Fractional-order gradient descent learning of BP neural networks with Caputo derivative

Wang, Jian; Wen, Yanqing; Gou, Yida; Ye, Zhenyun; Chen, Hua

doi:10.1016/j.neunet.2017.02.007

Cited by 153 publications

(86 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This idea is still novel and needs to see improvements. For example, the gradient descent method has been handled by Sheng et al; [32], [33], Wang et al ; [28], Wei et al; [9] and Bao et al [22]. These methods are still early in development.…”

Section: Resultsmentioning

confidence: 99%

“…Fractional-order methods have been used to investigate complex-valued neural networks in [24] and recurrent neural network models in [44]. In [28] and [22], gradients based on the Caputo fractional derivative are used to update parameters while integer order gradients are used to handle backpropagation allowing for simpler computation. The experiments therein are shown to improve the accuracy of the neural network's performance compared to integer-order methods while being equally costly.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A sigmoidal fractional derivative for regularization

Rezapour¹,

Sijuwade²,

Asaki

2020

AIMS Mathematics

View full text Add to dashboard Cite

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A sigmoidal fractional derivative for regularization

Rezapour¹,

Sijuwade²,

Asaki

2020

AIMS Mathematics

View full text Add to dashboard Cite

show abstract

“…Simulated system shown in Figures 5-7 is an 8-input and 1-output spiking neural system where the first four kernel functions (k (1) , k (2) , k (3) , k (4) ) undergo step changes and the other two kernels (k (5) , k (6) ) undergo slower gradual changes concurrently. To make the tracking task more difficult, the last two kernels (k (7) , k (8) ) are designed to be zeroes as redundant inputs to the spiking neural system. The transient changes for all step changing kernels occur at 400 s. In particular, the amplitudes of k (1) and k (2) are doubled, while the amplitudes of k (3) and k (4) are decreased by half.…”

Section: Simulation Studiesmentioning

confidence: 99%

“…Peak amplitudes of actual kernels (black) and estimated kernels (blue for SSPPF, green for sSSPPF, purple for dMGLV, and red for the proposed sMGLV model) across simulation time evolution in a time-varying system with a 1 s resolution. (4) 0.0911 0.0882 0.0521 0.0408 k (5) 0.0525 0.0529 0.0260 0.0217 k (6) 0.0405 0.0387 0.0222 0.0162 k (7) 0.0431 0.0000 0.0250 0.0000 k (8) 0.0467 0.0000 0.0371 0.0000 k (9) 0 …”

Section: Simulation Studiesmentioning

confidence: 99%

A Sparse Multiwavelet-Based Generalized Laguerre–Volterra Model for Identifying Time-Varying Neural Dynamics from Spiking Activities

Huang

et al. 2017

Entropy

View full text Add to dashboard Cite

Abstract:Modeling of a time-varying dynamical system provides insights into the functions of biological neural networks and contributes to the development of next-generation neural prostheses. In this paper, we have formulated a novel sparse multiwavelet-based generalized Laguerre-Volterra (sMGLV) modeling framework to identify the time-varying neural dynamics from multiple spike train data. First, the significant inputs are selected by using a group least absolute shrinkage and selection operator (LASSO) method, which can capture the sparsity within the neural system. Second, the multiwavelet-based basis function expansion scheme with an efficient forward orthogonal regression (FOR) algorithm aided by mutual information is utilized to rapidly capture the time-varying characteristics from the sparse model. Quantitative simulation results demonstrate that the proposed sMGLV model in this paper outperforms the initial full model and the state-of-the-art modeling methods in tracking performance for various time-varying kernels. Analyses of experimental data show that the proposed sMGLV model can capture the timing of transient changes accurately. The proposed framework will be useful to the study of how, when, and where information transmission processes across brain regions evolve in behavior.

show abstract

“…D. C. Huang and S. C. Xie [3] applied BP neural network in monitoring tailings dam settlement. In 2017, J. Wang [11], etc. proposed a fractional gradient descent method for the BP training of neural networks, and the monotonicity and weak (strong) convergence of the proposed approach are proved in detail.…”

Section: Introductionmentioning

confidence: 99%

A Study on Faults Diagnosis and Early-Warning Method of Tailings Reservoir Monitoring Points based on Intelligent Discovery

Wu¹

2017

IJPE

View full text Add to dashboard Cite

The tailings reservoir is a major hazard source with high potential energy, which may cause an artificial debris flow. The stability of the tailings reservoir is extremely important to the normal operation of the mining enterprises and the safety of people's lives and property. In order to settle the problem that traditional manual monitoring is scattered, not timely, and difficult to manage, this article takes Huangmailing tailings as an example, and establishes the CMST model to optimize the network topology connection of the tailings monitoring points. BP neural network algorithm is used to discuss the intelligent discovery and early warning of the faults on-line monitoring system of tailings. In this way, the fault-points and the causes can be perceived quickly and accurately, and the risk of the tailings' safety accident can be reduced. It can be proved by the experimental results and two years stable operation of the system that BP neural network algorithm can accurately predict the value of safety monitoring data.

show abstract

Fractional-order gradient descent learning of BP neural networks with Caputo derivative

Cited by 153 publications

References 18 publications

A sigmoidal fractional derivative for regularization

A sigmoidal fractional derivative for regularization

A Sparse Multiwavelet-Based Generalized Laguerre–Volterra Model for Identifying Time-Varying Neural Dynamics from Spiking Activities

A Study on Faults Diagnosis and Early-Warning Method of Tailings Reservoir Monitoring Points based on Intelligent Discovery

Contact Info

Product

Resources

About