Parallel Learning of Feedforward Neural Networks Without Error Backpropagation

Bilski, Jarosław; Wilamowski, Bogdan M.

doi:10.1007/978-3-319-39378-0_6

Cited by 14 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Algorithm 1: The GQR algorithm while the stopping criterion is not met do for each sample n do Perform network forward pass Perform error backpropagation Begin the GQR algorithm: for each layer l do for each neuron i do Compute equations ( 30), ( 31), (32) Begin the QR decomposition (35):…”

Section: Simulation Resultsmentioning

confidence: 99%

“…Our further research will focus on a parallel variant of the GQR algorithm similar to the methods presented in [35][36][37][38]. Also, the presented method is going to be attempted in the Convolutional Neural Networks (CNN) training.…”

Section: Discussionmentioning

confidence: 99%

“…This approach allows to solve the equation ( 27) easily and quickly with limited computational load. During the process Q (l)T i (n) matrix is implicitly calculated as shown in the following equations (35) where R i (n), whose inversion is not very complex. The approximated weights vectors are calculated according to the following equation…”

Section: Network Trainingmentioning

confidence: 99%

See 2 more Smart Citations

A Novel Fast Feedforward Neural Networks Training Algorithm

Bilski

Kowalczyk

Marjański

et al. 2021

Journal of Artificial Intelligence and Soft Computing Research

Self Cite

View full text Add to dashboard Cite

In this paper1 a new neural networks training algorithm is presented. The algorithm originates from the Recursive Least Squares (RLS) method commonly used in adaptive filtering. It uses the QR decomposition in conjunction with the Givens rotations for solving a normal equation - resulting from minimization of the loss function. An important parameter in neural networks is training time. Many commonly used algorithms require a big number of iterations in order to achieve a satisfactory outcome while other algorithms are effective only for small neural networks. The proposed solution is characterized by a very short convergence time compared to the well-known backpropagation method and its variants. The paper contains a complete mathematical derivation of the proposed algorithm. There are presented extensive simulation results using various benchmarks including function approximation, classification, encoder, and parity problems. Obtained results show the advantages of the featured algorithm which outperforms commonly used recent state-of-the-art neural networks training algorithms, including the Adam optimizer and the Nesterov’s accelerated gradient.

show abstract

Section: Simulation Resultsmentioning

confidence: 99%