Universal Approximation of Extreme Learning Machine With Adaptive Growth of Hidden Nodes

Abstract

Extreme learning machines (ELMs) have been proposed for generalized single-hidden-layer feedforward networks which need not be neuron-like and perform well in both regression and classification applications. In this brief, we propose an ELM with adaptive growth of hidden nodes (AG-ELM), which provides a new approach for the automated design of networks. Different from other incremental ELMs (I-ELMs) whose existing hidden nodes are frozen when the new hidden nodes are added one by one, in AG-ELM the number of hidden nodes is determined in an adaptive way in the sense that the existing networks may be replaced by newly generated networks which have fewer hidden nodes and better generalization performance. We then prove that such an AG-ELM using Lebesgue p-integrable hidden activation functions can approximate any Lebesgue p-integrable function on a compact input set. Simulation results demonstrate and verify that this new approach can achieve a more compact network architecture than the I-ELM.

A Survey on Machine Learning-Based Mobile Big Data Analysis: Challenges and Applications

“…There are usually two ways to solve this problem: (1) training ELM [114] based with divide-andconquer strategy; (2) introducing parallel mechanism [115] to train a single ELM. It is shown in [116,117] that a single ELM has strong function approximation ability. Whether it is possible to extend this approximation capability to ELM based on divide-and-conquer strategy is a key index to evaluate the possibility that ELM can be applied to big data.…”

Short-Term Electricity-Load Forecasting Using a TSK-Based Extreme Learning Machine with Knowledge Representation

“…Several studies have been investigated the development of good learning methods in the past decades. In contrast to well-known neural networks, e.g., the multilayer perceptron (MLP) and radial basis function network (RBFN), ELMs possess a real-time learning capability and good prediction ability as intelligent predictors [21,24,25]. Figure 1 shows the architecture of an ELM predictor.…”

“…First, the hidden layer does not need to be tuned. Second, the hidden layer mapping h(x) satisfies the universal approximation conditions [24]. Next, the ELM parameters are estimated to minimize, as follows:…”

An Effective Intelligent Self-Construction Multilayer Perceptron Neural Network

“…These methods are called constructive algorithms. The most common constructing algorithms are growing cell structure (GCS) [4], constructive back-propagation (CBP) [14], adaptively constructing multilayer FNN [16], extreme learning machine with adaptive growth of hidden nodes (AG-ELM) [23], enhanced incremental extreme learning machine (EI-ELM) [8], and etc. A hybrid approach which merges the constructive and pruning algorithms is also introduced.…”