Stable bagging feature selection on medical data

Abstract

In the medical field, distinguishing genes that are relevant to a specific disease, let’s say colon cancer, is crucial to finding a cure and understanding its causes and subsequent complications. Usually, medical datasets are comprised of immensely complex dimensions with considerably small sample size. Thus, for domain experts, such as biologists, the task of identifying these genes have become a very challenging one, to say the least. Feature selection is a technique that aims to select these genes, or features in machine learning field with respect to the disease. However, learning from a medical dataset to identify relevant features suffers from the curse-of-dimensionality. Due to a large number of features with a small sample size, the selection usually returns a different subset each time a new sample is introduced into the dataset. This selection instability is intrinsically related to data variance. We assume that reducing data variance improves selection stability. In this paper, we propose an ensemble approach based on the bagging technique to improve feature selection stability in medical datasets via data variance reduction. We conducted an experiment using four microarray datasets each of which suffers from high dimensionality and relatively small sample size. On each dataset, we applied five well-known feature selection algorithms to select varying number of features. The proposed technique shows a significant improvement in selection stability while at least maintaining the classification accuracy. The stability improvement ranges from 20 to 50 percent in all cases. This implies that the likelihood of selecting the same features increased 20 to 50 percent more. This is accompanied with the increase of classification accuracy in most cases, which signifies the stated results of stability.

An Explainable Machine Learning Approach for COVID-19’s Impact on Mood States of Children and Adolescents during the First Lockdown in Greece

“…The feature selection process was performed by using the ReliefF algorithm, due to its effectiveness in medical diagnosis and medical classification problems [ 43 , 44 , 45 , 46 , 47 ]. ReliefF is an extension of the original Relief which can deal with multiclass problems due to its enhancement with noise resistance [ 48 , 49 ], and therefore it is considered suitable for the current medical multiclass classification problem, as defined in Section 3.3 , Figure 2 .…”

A Real Estate Valuation Model Using Boosted Feature Selection

“…the ensemble learning based feature selection is the combination of several feature selection methods and ensemble learning to compensate the inconsistencies between elementary feature selectors and improve the robustness of selection process [30], [38], [54], [62]. This empirically enhance the selection robustness and overcome the approach with considerable stability improvement in several domains [2], [6], [42], [59].…”

Radiomics for Discrimination between Early-Stage Nasopharyngeal Carcinoma and Benign Hyperplasia with Stable Feature Selection on MRI

“…In light of this, methods to improve feature selection stability for medical imaging have rapidly gained attention. Several recent publications have highlighted the use of ensemble methods to improve the feature selection stability [ 16 , 17 , 18 , 19 ], mainly investigating three ensemble techniques, namely resampling, bagging, and boosting; however, their combined use has not been studied in radiomics. It is possible that using these techniques in combination could further improve the stability.…”