A Fast SVD-Hidden-nodes based Extreme Learning Machine for Large-Scale Data Analytics

Abstract

Big dimensional data is a growing trend that is emerging in many real world contexts, extending from web mining, gene expression analysis, protein-protein interaction to high-frequency financial data. Nowadays, there is a growing consensus that the increasing dimensionality poses impeding effects on the performances of classifiers, which is termed as the "peaking phenomenon" in the field of machine intelligence. To address the issue, dimensionality reduction is commonly employed as a preprocessing step on the Big dimensional data before building the classifiers. In this paper, we propose an Extreme Learning Machine (ELM) approach for large-scale data analytic. In contrast to existing approaches, we embed hidden nodes that are designed using singular value decomposition (SVD) into the classical ELM. These SVD nodes in the hidden layer are shown to capture the underlying characteristics of the Big dimensional data well, exhibiting excellent generalization performances. The drawback of using SVD on the entire dataset, however, is the high computational complexity involved. To address this, a fast divide and conquer approximation scheme is introduced to maintain computational tractability on high volume data. The resultant algorithm proposed is labeled here as Fast Singular Value Decomposition-Hidden-nodes based Extreme Learning Machine or FSVD-H-ELM in short. In FSVD-H-ELM, instead of identifying the SVD hidden nodes directly from the entire dataset, SVD hidden nodes are derived from multiple random subsets of data sampled from the original dataset. Comprehensive experiments and comparisons are conducted to assess the FSVD-H-ELM against other state-of-the-art algorithms. The results obtained demonstrated the superior generalization performance and efficiency of the FSVD-H-ELM.

Neural network-based formula for shear capacity prediction of one-way slabs under concentrated loads

“…Although Deng et al (2016) proposed this method for a 3-layer network, it was implemented in this work regardless the number of hidden layers.…”

“…Based on Deng et al (2016), this scheme is an alternative version of the former SVD. Now, training data is split into min{Qb, Pt} chunks (or subsets) of equal size Pti = max{floor(Pt / Qb), 1} -"floor" rounds the argument to the previous integer (whenever it is decimal) or yields the argument itself, being each chunk aimed to derive Qbi = 1 hidden node.…”

Potential of neural networks for structural damage localization

“…Although Deng et al [45] proposed this method for a 3-layer network, it was implemented in this work regardless the number of hidden layers.…”

“…Based on [45], this scheme is an alternative version of the former SVD. Now, training data is split into min {Q b , P t } chunks (or subsets) of equal size P ti = max {floor(P t / Q b ), 1}floor rounds the argument to the previous integer (whenever it is decimal) or yields the argument itself, being each chunk aimed to derive Q bi = 1 hidden node.…”

Domain Adaption Based on ELM Autoencoder

“…Just like the traditional ELM [21][22][23], ELM-AE contains three layers: input layer, hidden layer, and output layer. The difference is that the target output is the same as the input in ELM-AE.…”