First Order Statistics Based Feature Selection: A Diverse and Powerful Family of Feature Seleciton Techniques

2014 IEEE International Conference on Bioinformatics and Bioengineering

Napolitano

2014

One of the more prevalent problems when working with bioinformatics datasets is class imbalance, when there are more instances in one class compared to the other class(es). This problem is made worse because frequently, the class of interest is also the minority class. A possible solution is data sampling, a powerful tool for combating class imbalance by adding or removing instances to make the dataset more balanced. In addition to the choice of including data sampling, one of the most important decisions when applying data sampling is what the final class ratio should be. Commonly, the final class ratio when data sampling is applied is 50:50, however it is an open question whether other ratios are more appropriate for certain imbalanced datasets (all datasets in this paper have 25.16% minority instances or less) where a 50:50 ratio requires extreme modification to the dataset. In this work we compare six different data sampling approaches (feature selection with the pairwise combinations of three data sampling techniques and two final class ratios) with feature selection without data sampling with the goal of determining if the inclusion of data sampling is beneficial and if so, what should be the final class ratio. In order to test the six data sampling approaches and feature selection alone thoroughly, we utilize seven imbalanced and high-dimensional datasets, three feature selection techniques, and six classifiers. Our results show that for a majority of scenarios, random undersampling along with either 35:65 or 50:50 is the best data sampling approach. Statistical analysis shows that there is no significant difference between the data sampling approaches. However, despite this, we still recommend using random undersampling along with 35:65 as the final class ratio. This is because of the frequency of both random undersampling and 35:65 being the most frequent top performing data sampling technique and class ratio respectively. Additionally, 35:65 will have fewer negative impacts than 50:50 (less data loss or overfitting, which makes it a better choice if all other factors are equal) and random undersampling is more computationally efficient than any other form of sampling, including "no sampling" (both by not requiring any internal calculations and by producing a reduced, easier-to-work-with dataset). To our knowledge, this is the most comprehensive work which focuses on the choice of the inclusion and implementation of data sampling with different final class ratios on bioinformatics datasets which exhibit such large levels of class imbalance.

Section: B Feature Selection Techniquesmentioning

confidence: 99%

“…When using feature selection a final feature subset size must be chosen. In our work we use 25 features which, based on previous research, is a reasonable number of features [15].…”

Section: B Feature Selection Techniquesmentioning

confidence: 99%

Selecting the Appropriate Data Sampling Approach for Imbalanced and High-Dimensional Bioinformatics Datasets

2014 IEEE International Conference on Bioinformatics and Bioengineering

Napolitano

2014

“…Wrappers, unlike filter approaches, use classifiers when making a decision, and often the classifier used to calculate the score of a particular feature subset is the same one that will be used in the post selection analysis. There are two main disadvantages in the use of wrapper based feature selection techniques: limited utility of chosen features (due to the chosen features being specific to the learner used within the wrapper and thus not necessarily important to the problem itself) and slow computation time (from For more information regarding the FOS techniques refer to Khoshgoftaar et al [11].…”

Section: B Feature Selection Techniquesmentioning

confidence: 99%

Simplifying the Utilization of Machine Learning Techniques for Bioinformatics

2013 12th International Conference on Machine Learning and Applications

Wald

et al. 2013

Self Cite

The domain of bioinformatics has a number of challenges such as handling datasets which exhibit extreme levels of high dimensionality (large number of features per sample) and datasets which are particularly difficult to work with. These datasets contain many pieces of data (features) which are irrelevant and redundant to the problem being studied, which makes analysis quite difficult. However, techniques from the domain of machine learning and data mining are well suited to combating these difficulties. Techniques like feature selection (choosing an optimal subset of features for subsequent analysis by removing irrelevant or redundant features) and classifiers (used to build inductive models in order to classify unknown instances) can assist researchers in working with such difficult datasets. Unfortunately, many practitioners of bioinformatics do not have the machine learning knowledge to choose the correct techniques in order to achieve good classification results. If the choices could be simplified or predetermined then it would be easier to apply the techniques. This study is a comprehensive analysis of machine learning techniques on twenty-five bioinformatics datasets using six classifiers, and twenty-four feature rankers. We analyzed the factors at each of four feature subset sizes chosen for being large enough to be effective in creating inductive models but small enough to be of use for further research. Our results shows that Random Forest with 100 trees is the top performing classifier and that the choice of feature ranker is of little importance as long as feature selection occurs. Statistical analysis confirms our results. By choosing these parameters, machine learning techniques are more accessible to bioinformatics.

“…Once the ranking is complete, the user can choose a subset of the top performing features for use in subsequent analysis. In this work, we use four rankers from three different families of filter-based feature selection methods: two rankers, Information Gain (IG) and ReleifF (RF), are from the "Commonly-Used" family, one ranker, Area Under the ROC Curve (ROC), is from the "ThresholdBased Feature Selection" (TBFS) family, and Signal-to-Noise (S2N) that is a ranking technique from the family of FirstOrder Statistics-based (FOS) [9] . We use rankers because filter-and wrapper-based subset selection techniques can be computationally prohibitive in particular for datasets with high number of features (genes) that are very common in the field of bioinformatics.…”

Section: Feature Selection Techniquesmentioning

confidence: 99%

Classification performance of three approaches for combining data sampling and gene selection on bioinformatics data

Fazelpour

Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration (IEEE IRI 2014)

et al. 2014

Self Cite

Bioinformatics datasets pose two major challenges to researchers and data-mining practitioners: class imbalance and high dimensionality. Class imbalance occurs when instances of one class vastly outnumber instances of the other class(es), and high dimensionality occurs when a dataset has many independent features (genes). Data sampling is often used to tackle the problem of class imbalance, and the problem of excessive features in the dataset may be alleviated through feature selection. In this work, we examine various approaches for applying these techniques simultaneously to tackle both of these challenges and build effective classification models. In particular, we ask whether the order of these techniques and the use of unsampled or sampled datasets for building classification models makes a difference. We conducted an empirical study on a series of seven highdimensional and severely imbalanced biological datasets using six commonly used learners and four feature selection rankers from three different families of feature selection techniques. We compared three different data-sampling approaches: data sampling followed by feature selection using the unsampled data (DS-FS-UnSam) and selected features; data sampling followed by feature selection using the sampled data (DS-FS-Sam) and selected features; and feature selection followed by data sampling (FS-DS) using sampled data and selected features. We used Random Undersampling (RUS) to achieve the minority:majority class ratios of 35:65 and 50:50. The experimental results show that there are statistically significant differences among the three data-sampling approaches only when using the class ratio of 50:50, with a multiple comparison test showing that DS-FS-UnSam outperforms the other approaches. Thus, although specific combinations of learner and ranker may favor other approaches, across all choices of learner and ranker we would recommend the use of the DS-FS-UnSam approach for this class ratio. On the other hand, with the 35:65 class ratio, DS-FSSam was most frequently the top-performing approach, and although it was not statistically significantly better than the other approaches, we would generally recommend this approach be used for the 35:65 class ratio (although specific choices of learner and ranker may vary). Overall, we can see that the optimal approach will depend on the choice of class ratio.