A double-layer ELM with added feature selection ability using a sparse Bayesian approach

Kiaee, Farkhondeh; Gagné, Christian; Sheikhzadeh, Hamid

doi:10.1016/j.neucom.2016.08.011

Cited by 6 publications

(2 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sparse learning has been shown to be efficient at pruning the irrelevant parameters in many practical applications, by incorporating sparsity-promoting penalty functions into the original problem, where the added sparsity-promoting terms penalize the number of parameters (Kiaee et al (2016a;b;c)). Motivated by learning efficient architectures of a deep CNN for embedded implementations, our work focuses on the design of a sparse network using an initial pre-trained dense CNN.…”

Section: Introductionmentioning

confidence: 99%

Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networks

Kiaee,

Gagné,

Abbasi

2016

Preprint

Self Cite

View full text Add to dashboard Cite

The storage and computation requirements of Convolutional Neural Networks (CNNs) can be prohibitive for exploiting these models over low-power or embedded devices. This paper reduces the computational complexity of the CNNs by minimizing an objective function, including the recognition loss that is augmented with a sparsity-promoting penalty term. The sparsity structure of the network is identified using the Alternating Direction Method of Multipliers (ADMM), which is widely used in large optimization problems. This method alternates between promoting the sparsity of the network and optimizing the recognition performance, which allows us to exploit the two-part structure of the corresponding objective functions. In particular, we take advantage of the separability of the sparsity-inducing penalty functions to decompose the minimization problem into sub-problems that can be solved sequentially. Applying our method to a variety of state-of-the-art CNN models, our proposed method is able to simplify the original model, generating models with less computation and fewer parameters, while maintaining and often improving generalization performance. Accomplishments on a variety of models strongly verify that our proposed ADMM-based method can be a very useful tool for simplifying and improving deep CNNs.

show abstract

Section: Introductionmentioning

confidence: 99%

Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networks

Kiaee,

Gagné,

Abbasi

2016

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Assis Boldt et al 36 adopted three cascade combinations and extreme learning machine (ELM) for fault diagnosis. Kiaee et al 37 proposed a sparse Bayesian Double-Layer ELM to deal with high-dimensional data.…”

Section: Introductionmentioning

confidence: 99%

Multi-feature learning-based extreme learning machine for rolling bearing fault diagnosis

Zheng

Xiang

Cui

2021

Proceedings of the Institution of Mechanical Engineers, Part O:

View full text Add to dashboard Cite

Rolling bearing has been becoming an important part of human life and work. The working environment of rolling bearing is very complex and variable, which makes it difficult for fault diagnosis and monitor of rolling bearing from raw vibration data. Then, in this paper, a novel multi-feature learning-based extreme learning machine is proposed for rolling bearing fault diagnosis (FL-ELM). Extreme learning machine (ELM) is a fast and generalized algorithm proposed for training single-hidden-layer feed-forward networks (SLFNs), which has fast computing speed and small testing error. The novel architecture has two hidden layers and an experience pool sandwiched between two hidden layers. The first hidden layer consists of multi-feature learning methods. The experience pool is used to sort and choose new data, with old data being filtered out. Firstly, the first hidden layer is adopted for feature extraction. Secondly, the experience pool is used to rearrange and select data, which is extracted by first hidden layer. Thirdly, ELM is employed to further learn and classify. The proposed method (FL-ELM) is applied to the rolling bearing fault diagnosis. The results confirm that the proposed method is more effective than traditional methods and standard deep learning methods.

show abstract

Bayesian extreme learning machines for hydrological prediction uncertainty

Quilty,

Jahangir,

You

et al. 2023

Journal of Hydrology

View full text Add to dashboard Cite

A double-layer ELM with added feature selection ability using a sparse Bayesian approach

Cited by 6 publications

References 23 publications

Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networks

Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networks

Multi-feature learning-based extreme learning machine for rolling bearing fault diagnosis

Bayesian extreme learning machines for hydrological prediction uncertainty

Contact Info

Product

Resources

About