Multilabel Feature Selection Using Relief and Minimum Redundancy Maximum Relevance Based on Neighborhood Rough Sets

Abstract

Recently, multilabel classification is of increasing interest in machine learning and artificial intelligence. However, the distances of samples in most Relief methods easily result in heterogeneous or similar samples abnormal when the distances are very large. Besides, the classification margin as a neighborhood radius for some reduction algorithms may be meaningless when the margin is too large. To overcome these drawbacks, this paper presents a multilabel feature selection method using the improved Relief and minimum redundancy maximum relevance (MRMR) based on neighborhood rough sets. First, the number of heterogeneous and similar samples is introduced to improve the label weighting method which can eliminate the influence of the large distances of samples. By combining with the new label weighting, the distances between the sample and its nearest-neighbor heterogeneous sample and between the sample and its nearest-neighbor similar sample are presented to develop a new feature weighting method. Second, the number of heterogeneous and similar samples continues to be used to improve the classification margin, thereby constraining the neighborhood radius, based on which the neighborhood approximation accuracy is constructed to effectively measure the uncertainty of samples in the boundary region and the completeness of knowledge. Third, by integrating with the new neighborhood approximation accuracy, two types of mutual information between features and labels and among features are proposed, and then the mutual informationbased MRMR model is investigated to evaluate the significance of features. Finally, a multilabel feature selection algorithm is designed for improving the classification performance of multilabel data. Experimental results on thirteen public datasets illustrate the effectiveness of our developed algorithm that can select the significant features and achieve great performance for multilabel datasets.

Multilabel Feature Selection Using Mutual Information and ML-ReliefF for Multilabel Classification

“…Similarly, the appropriate connection parameters on the Health, Computers, Yeast, Business, and Reference datasets have been achieved in Section IV.B. Following the experiments provided in [3], [43], [49], the five datasets are selected from Table 3 for comparison in this experiment. The values of the six indices on five multilabel datasets are shown in Tables 8-12, respectively. As seen from Table 8, MFS-MIRF achieves the best values of the AP, HL, and OE indices.…”

“…According to the experiments [45], a classifier is set through the simplified training datasets, and the classification results are usually implemented on the simplified test datasets. Therefore, these datasets in Table 3 will be divided into training datasets and test datasets, and the training datasets are used for feature selection, and the test datasets evaluate the classification performance of various methods on the different classifier [3] including MLKNN, Lazy, Rules, Trees, and Bayes. To evaluate the effectiveness of our algorithm in multilabel classification, six metrics are employed as follows: the number of features selected (N), Average Precision (AP), Hamming Loss (HL), Ranking Loss (RL), One Error (OE), and Coverage (CV) [2].…”

“…As an important preprocessing part of multilabel classification, feature selection aims to remove redundant or irrelevant features, eliminate the curse of dimensionality, classify data and obtain useful information to improve classification performance [2]. Feature selection models for multilabel classification often can be divided into three strategies: filter, wrapper and embedded methods [3]. Filter methods are independent of learning algorithms and the specified classifier for selecting feature subsets [4].…”

“…The Relief algorithm presented by kria and Rendell was originally used for two-category problems [3]. To adapt to multiple category problems, Kononenko [20] proposed the ReliefF algorithm based on Relief to deal with multilabel data, but did not take full advantage of correlation degree among labels.…”

“…By considering the ratios of different labels, the correlation degree of features and label sets is calculated to preprocess multilabel data for reducing the runtime of ML-ReliefF. 3 (2) To overcome the possible error caused by the popular coefficient on correlation degree among label sets, mutual information among label sets is introduced into the correlation degree of label sets to discover nonlinear relationships among labels. In order to divide the similar and heterogeneous samples more clearly, two types of correlation degree of label sets based on ML-ReliefF are developed.…”