Artificial neural networks training acceleration through network science strategies

Abstract

The development of deep learning has led to a dramatic increase in the number of applications of artificial intelligence. However, the training of deeper neural networks for stable and accurate models translates into artificial neural networks (ANNs) that become unmanageable as the number of features increases. This work extends our earlier study where we explored the acceleration effects obtained by enforcing, in turn, scale freeness, small worldness, and sparsity during the ANN training process. The efficiency of that approach was confirmed by recent studies (conducted independently) where a million-node ANN was trained on non-specialized laptops. Encouraged by those results, our study is now focused on some tunable parameters, to pursue a further acceleration effect. We show that, although optimal parameter tuning is unfeasible, due to the high non-linearity of ANN problems, we can actually come up with a set of useful guidelines that lead to speed-ups in practical cases. We find that significant reductions in execution time can generally be achieved by setting the revised fraction parameter ($$\zeta $$ ζ ) to relatively low values.

Adaptive Momentum-Backpropagation Algorithm for Flood Prediction and Management in the Internet of Things

“…To ensure the success of the training, different parts of the data must be examined. The training efficiency of neural network models is dependent on the data and the learning algorithm adopted [47,48]. Advanced learning algorithms effectively transfer intelligence to the ANN from historical facts or data.…”

Yapay Sinir Ağları ve K-Ortalamalar Tabanlı Büyük Veri Azaltma Algoritmasının Tasarımı ve Uygulaması

“…YSA'nın diğer bir temel öğesi olan Aktivasyon Fonksiyonu bir değişkeni farklı bir boyuta taşıyan doğrusal veya doğrusal olmayan bir fonksiyondur. YSA'da Aktivasyon Fonksiyonu yapay sinir hücresi girdi verileri üzerinde işlem yaparak buna karşılık gelen net çıktı sonuçları elde eder [13][14][15][16][17][18][19][20].…”

To the special issue dedicated to the 3rd international conference “Numerical Computations: Theory and Algorithms—NUMTA 2019” June 15–21, 2019, Isola Capo Rizzuto, Italy

“…Two main partially overlapping groups of papers can be distinguished within it. The first group discusses optimization problems and algorithms including local, global, and multi-criteria optimization (see Franchini et al 2020;Ž ilinskas and Litvinas 2020;Nesterov 2020;Posypkin et al 2020;Shao et al 2020;De Leone et al 2020;Capuano et al 2020;Lančinskas et al 2020;Sergeyev et al 2020;Crisci et al 2020;Cavallaro et al 2020;Astorino and Fuduli 2020;Candelieri et al 2020). The second group of papers (see D'Alotto 2020; Pepelyshev and Zhigljavsky 2020;Falcone et al 2020;Amodio et al 2020;Gangle et al 2020;De Leone et al 2020;Astorino and Fuduli 2020) deals with problems and algorithms using the already mentioned recent computational framework allowing one to work with different infinities and infinitesimals numerically.…”