Parallel methods for linear systems solution in extreme learning machines: an overview

Gelvez-Almeida, E; Baldera-Moreno, Y; Huérfano, Yoleidy; Vera, Matías; Mora, Marco; Barrientos, Ricardo J.

doi:10.1088/1742-6596/1702/1/012017

Cited by 3 publications

(1 citation statement)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reported works are classified as follows: (1) methods based on matrix operations, (2) methods based on matrix decomposition, and (3) iterative methods. This section is an update of the review in a previous study [59]. The works presented are related to the algebraic processes associated with ELM models and their variants.…”

Section: Methods For Solving Linear Systems With Parallel and Distrib...mentioning

confidence: 98%

A Review on Large-Scale Data Processing with Parallel and Distributed Randomized Extreme Learning Machine Neural Networks

Gelvez-Almeida,

Mora,

Barrientos

et al. 2024

MCA

View full text Add to dashboard Cite

The randomization-based feedforward neural network has raised great interest in the scientific community due to its simplicity, training speed, and accuracy comparable to traditional learning algorithms. The basic algorithm consists of randomly determining the weights and biases of the hidden layer and analytically calculating the weights of the output layer by solving a linear overdetermined system using the Moore–Penrose generalized inverse. When processing large volumes of data, randomization-based feedforward neural network models consume large amounts of memory and drastically increase training time. To efficiently solve the above problems, parallel and distributed models have recently been proposed. Previous reviews of randomization-based feedforward neural network models have mainly focused on categorizing and describing the evolution of the algorithms presented in the literature. The main contribution of this paper is to approach the topic from the perspective of the handling of large volumes of data. In this sense, we present a current and extensive review of the parallel and distributed models of randomized feedforward neural networks, focusing on extreme learning machine. In particular, we review the mathematical foundations (Moore–Penrose generalized inverse and solution of linear systems using parallel and distributed methods) and hardware and software technologies considered in current implementations.

show abstract

Section: Methods For Solving Linear Systems With Parallel and Distrib...mentioning

confidence: 98%

A Review on Large-Scale Data Processing with Parallel and Distributed Randomized Extreme Learning Machine Neural Networks

Gelvez-Almeida,

Mora,

Barrientos

et al. 2024

MCA

View full text Add to dashboard Cite

show abstract

A Review of multilayer extreme learning machine neural networks

2023

View full text Add to dashboard Cite

Parallel ensemble of a randomization-based online sequential neural network for classification problems using a frequency criterion

Gelvez-Almeida,

Barrientos,

Vilches-Ponce

et al. 2024

Sci Rep

View full text Add to dashboard Cite

Randomization-based neural networks have gained wide acceptance in the scientific community owing to the simplicity of their algorithm and generalization capabilities. Random vector functional link (RVFL) networks and their variants are a class of randomization-based neural networks. RVFL networks have shown promising results in classification, regression, and clustering problems. For real-world applications, learning algorithms that can train with new samples over previous results are necessary because of to the constant generation of problems related to large-scale datasets. Various online sequential algorithms, commonly involving an initial learning phase followed by a sequential learning phase, have been proposed to address this issue. This paper presents a training algorithm based on multiple online sequential random vector functional link (OS-RVFL) networks for large-scale databases using a shared memory architecture. The training dataset is distributed among p OS-RVFL networks, which are trained in parallel using p threads. Subsequently, the test dataset samples are classified using each trained OS-RVFL network. Finally, a frequency criterion is applied to the results obtained from each OS-RVFL network to determine the final classification. Additionally, an equation was derived to reasonably predict the total training time of the proposed algorithm based on the learning time in the initial phase and the time scaling factor compared to the sequential learning phase. The results demonstrate a drastic reduction in training time because of data distribution and an improvement in accuracy because of the adoption of the frequency criterion.

show abstract

Parallel methods for linear systems solution in extreme learning machines: an overview

Cited by 3 publications

References 23 publications

A Review on Large-Scale Data Processing with Parallel and Distributed Randomized Extreme Learning Machine Neural Networks

A Review on Large-Scale Data Processing with Parallel and Distributed Randomized Extreme Learning Machine Neural Networks

A Review of multilayer extreme learning machine neural networks

Parallel ensemble of a randomization-based online sequential neural network for classification problems using a frequency criterion

Contact Info

Product

Resources

About