Extreme Learning Machines with Simple Cascades

Abstract

We compare extreme learning machines with cascade correlation on a standard benchmark dataset for comparing cascade networks along with another commonly used dataset. We introduce a number of hybrid cascade extreme learning machine topologies ranging from simple shallow cascade ELM networks to full cascade ELM networks. We found that the simplest cascade topology provided surprising benefit with a cascade correlation style cascade for small extreme learning machine layers. Our full cascade ELM architecture achieved high performance with even a single neuron per ELM cascade, suggesting that our approach may have general utility, though further work needs to be done using more datasets. We suggest extensions of our cascade ELM approach, with the use of network analysis, addition of noise, and unfreezing of weights.

Bio-inspired computing tools and applications: position paper

“…Each ELM hidden neuron can be considered to come with some initial (random) fixed functionality and the training of the output weights is a weighted selection from the available menu. A small ELM network could be used as a higher order neuron, and some initial work has been done [14] (see Fig. 5).…”