FeaSel-Net: A Recursive Feature Selection Callback in Neural Networks

Abstract

Selecting only the relevant subsets from all gathered data has never been as challenging as it is in these times of big data and sensor fusion. Multiple complementary methods have emerged for the observation of similar phenomena; oftentimes, many of these techniques are superimposed in order to make the best possible decisions. A pathologist, for example, uses microscopic and spectroscopic techniques to discriminate between healthy and cancerous tissue. Especially in the field of spectroscopy in medicine, an immense number of frequencies are recorded and appropriately sized datasets are rarely acquired due to the time-intensive measurements and the lack of patients. In order to cope with the curse of dimensionality in machine learning, it is necessary to reduce the overhead from irrelevant or redundant features. In this article, we propose a feature selection callback algorithm (FeaSel-Net) that can be embedded in deep neural networks. It recursively prunes the input nodes after the optimizer in the neural network achieves satisfying results. We demonstrate the performance of the feature selection algorithm on different publicly available datasets and compare it to existing feature selection methods. Our algorithm combines the advantages of neural networks’ nonlinear learning ability and the embedding of the feature selection algorithm into the actual classifier optimization.

Data-driven development of sparse multi-spectral sensors for urological tissue differentiation

“…When speaking of features in this context, wavenumbers are meant. The results of classical multivariate statistical approaches using principal component analysis (PCA) [9,11,12] and linear discriminant analysis (LDA) [13] are compared to those of a recently developed FS algorithm called FeaSel-Net [10], that is based on recursive elimination of the input signal in neural networks. Whereas both former analyses are based on linear transformations and covariance evaluations, the latter is inherently non-linear and promises more complex solutions for the selection.…”

“…Another evaluation of the wavenumbers is done using FeaSel-Net [10]. This is a non-linear recursive FS method, that has recently been developed and already been used in several analyses of Raman and IR spectra [9,10,23]. It can be embedded in any 1D neural network.…”

“…For a first estimate of the discriminability of human tissue with an extremly sparsified infrared (IR) spectrum, we measured urothelia and urothelial carcinoma of bladder entities. By utilizing several different data-driven feature selection (FS) methods [9][10][11][12][13], we show that only a small fraction of the original MIR spectrum is necessary to reliably separate benign from malignant urothelia.…”