Extreme Learning Machine for Multilayer Perceptron

Tang, J. K. K.; Deng, Chenwei; Huang, Guang-Bin

doi:10.1109/tnnls.2015.2424995

Cited by 1,234 publications

(631 citation statements)

References 28 publications

Supporting

Mentioning

622

Contrasting

Unclassified

Order By: Relevance

“…Model DER% MLP [5] 1.60 MLP + dropout [11] 1.05 DBN [12] 1.03 DBM [13] 0.95 CNN [3] 0.95 MLP + maxout + dropout [8] 0.94 ELM [14] 0.86 RCN (This work) 0.81 DBM + dropout [7] 0.79 CNN + maxout + dropout [8] 0.45 a richer feature space for the final classification of the frames.…”

Section: Scanning Directionsmentioning

confidence: 99%

“…System DER% DER% (Original training set) (Enriched training set) LSTM [9,10] 1.80 0.32 2-layer MLP [5] 1.60 0.70 MLP + dropout [11] 1.05 -DBN [12] 1.03 -DBM [13] 0.95 -CNN [3] 0.95 0.80 MLP + maxout + dropout [8] 0.94 -ELM [14] 0.86 -DCN [15,16] 0.83 0.35 DBM + dropout [7] 0.79 -Large CNN [17] 0.60 0.39 CNN + maxout + dropout [8] 0.45 -Multi-CNN [18] -0.23 ture classification [9,10,22]. In this paper, the focus is on reservoir computing networks (RCNs) [23], which are a special type of recurrent neural networks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the application of reservoir computing networks for noisy image recognition

et al. 2018

View full text Add to dashboard Cite

Reservoir Computing Networks (RCNs) are a special type of single layer recurrent neural networks, in which the input and the recurrent connections are randomly generated and only the output weights are trained. Besides the ability to process temporal information, the key points of RCN are easy training and robustness against noise. Recently, we introduced a simple strategy to tune the parameters of RCNs. Evaluation in the domain of noise robust speech recognition proved that this method was effective. The aim of this work is to extend that study to the field of image processing, by showing that the proposed parameter tuning procedure is equally valid in the field of image processing and conforming that RCNs are apt at temporal modeling and are robust with respect to noise. In particular, we investigate the potential of RCNs in achieving competitive performance on the well-known MNIST dataset by following the aforementioned parameter optimizing strategy. Moreover, we achieve good noise robust recognition by utilizing such a network to denoise images and supplying them to a recognizer that is solely trained on clean images. The experiments demonstrate that the proposed RCN-based handwritten digit recognizer achieves an error rate of 0.81 percent on the clean test data of the MNIST benchmark and that the proposed RCN-based denoiser can effectively reduce the error rate on the various types of noise.

show abstract

Section: Scanning Directionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

On the application of reservoir computing networks for noisy image recognition

et al. 2018

View full text Add to dashboard Cite

show abstract

“…In contrast to deep network paradigm, an extreme learning machine (ELM) based hierarchical learning framework (H-ELM) proposed in [219] claimed a faster learning than deep learning by ELM [220] based auto-encoding. The proposed H-ELM framework worked in two phases: unsupervised hierarchical feature representation and 2) supervised feature classification [219].…”

Section: Architecture Plus Weight Optimizationmentioning

confidence: 99%

“…The proposed H-ELM framework worked in two phases: unsupervised hierarchical feature representation and 2) supervised feature classification [219].…”

Section: Architecture Plus Weight Optimizationmentioning

confidence: 99%

Metaheuristic design of feedforward neural networks: A review of two decades of research

Ojha

Abraham

Snel

2017

Engineering Applications of Artificial Intelligence

441

154

View full text Add to dashboard Cite

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era.

show abstract

“…The objective is to produce actions that maximize accumulated scores. This paper proposes a deep model that is based on Hierarchical Extreme learning machine [8] in a visual reinforcement learning task. The combination of deep learning and Reinforcement learning is very fruitful to have a complete system that gets visual data as input and gives optimal policy as output.…”

Section: Introductionmentioning

confidence: 99%

Hierarchical extreme learning machine based reinforcement learning for goal localization

AlDahoul

Htike

Akmeliawati

2017

IOP Conf. Ser.: Mater. Sci. Eng.

View full text Add to dashboard Cite

Localization and the connection between Uq(so(3)) and Uq(osp(1|2) Abstract. The objective of goal localization is to find the location of goals in noisy environments. Simple actions are performed to move the agent towards the goal. The goal detector should be capable of minimizing the error between the predicted locations and the true ones. Few regions need to be processed by the agent to reduce the computational effort and increase the speed of convergence. In this paper, reinforcement learning (RL) method was utilized to find optimal series of actions to localize the goal region. The visual data, a set of images, is high dimensional unstructured data and needs to be represented efficiently to get a robust detector. Different deep Reinforcement models have already been used to localize a goal but most of them take long time to learn the model. This long learning time results from the weights fine tuning stage that is applied iteratively to find an accurate model. Hierarchical Extreme Learning Machine (H-ELM) was used as a fast deep model that doesn't fine tune the weights. In other words, hidden weights are generated randomly and output weights are calculated analytically. H-ELM algorithm was used in this work to find good features for effective representation. This paper proposes a combination of Hierarchical Extreme learning machine and Reinforcement learning to find an optimal policy directly from visual input. This combination outperforms other methods in terms of accuracy and learning speed. The simulations and results were analysed by using MATLAB.

show abstract

Extreme Learning Machine for Multilayer Perceptron

Cited by 1,234 publications

References 28 publications

On the application of reservoir computing networks for noisy image recognition

On the application of reservoir computing networks for noisy image recognition

Metaheuristic design of feedforward neural networks: A review of two decades of research

Hierarchical extreme learning machine based reinforcement learning for goal localization

Contact Info

Product

Resources

About