Feature selection for multi-label classification using multivariate mutual information

Lee, Jae-Sung; Kim, Dae‐Won

doi:10.1016/j.patrec.2012.10.005

Cited by 251 publications

(158 citation statements)

References 15 publications

Supporting

Mentioning

156

Contrasting

Unclassified

Order By: Relevance

“…These two datasets are used by both Doquire & Verleysen [4] and Lee & Kim [9] to evaluate their criteria, with which we compare our own in Section 5. Table 2 summarises some characteristics of these datasets.…”

Section: Empirical Comparison Of the Assumptions In The Label Spacementioning

confidence: 99%

“…We compare J Y:full X:partial , the criterion with the best performance under our analysis, with two different criteria proposed recently in the literature: the pruned transformation criterion proposed by Doquire & Verleysen [4] (we prune rare examples using thresholds given in that work) and the multi-variate mutual information criterion proposed by Lee & Kim [9]. As we can see in Figure 5 the proposed criterion J Y:full X:partial consistently performs well across the different number of selected features and the different datasets.…”

Section: Comparison To the State-of-the-artmentioning

confidence: 99%

“…More particularly the present work focuses on information theoretic feature selection techniques in multi-label datasets, a problem that has recently received a lot of attention [4,10,9]. The starting point of our work is a recently proposed framework for single label data by Brown et al [3], which shows that many existing criteria can be seen as iterative maximizers of a common objective function: the conditional likelihood of the true label given the selected features.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Information Theoretic Feature Selection in Multi-label Data through Composite Likelihood

Sechidis

Νικολάου

Brown

2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In this paper we present a framework to unify information theoretic feature selection criteria for multi-label data. Our framework combines two different ideas; expressing multi-label decomposition methods as composite likelihoods and then showing how feature selection criteria can be derived by maximizing these likelihood expressions. Many existing criteria, until now proposed as heuristics, can be reproduced from a single basis under the proposed framework. Furthermore we can derive new problem-specific criteria by making different independence assumptions over the feature and label spaces. One such derived criterion is shown experimentally to outperform other approaches proposed in the literature on real-world datasets.

show abstract

Section: Empirical Comparison Of the Assumptions In The Label Spacementioning

confidence: 99%

Section: Comparison To the State-of-the-artmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Information Theoretic Feature Selection in Multi-label Data through Composite Likelihood

Sechidis

Νικολάου

Brown

2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Multi-label feature selection is considered a solution that can effectively avoid the aforementioned problems [5], [6]. Conventional multi-label feature selection methods evaluate the importance of each feature independently; therefore, the dependencies among features are ignored [2].…”

Section: Introudctionmentioning

confidence: 99%

“…Conventional multi-label feature selection methods evaluate the importance of each feature independently; therefore, the dependencies among features are ignored [2]. As a result, a compact multi-label feature subset cannot be obtained because a selected feature subset will necessarily contain redundant features, that is, features that are similar to one another [6]. To resolve this practical problem, a multi-label feature selection method must consider the feature dependencies during its feature selection process.…”

mentioning

confidence: 99%

Accelerating Multi-Label Feature Selection Based on Low-Rank Approximation

Lim

Lee

Kim

2016

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

SUMMARYWe propose a multi-label feature selection method that considers feature dependencies. The proposed method circumvents the prohibitive computations by using a low-rank approximation method. The empirical results acquired by applying the proposed method to several multilabel datasets demonstrate that its performance is comparable to those of recent multi-label feature selection methods and that it reduces the computation time. key words: multi-label feature selection, multivariate feature selection, feature dependency, Nyström method IntroudctionRecently, with the advancement of multi-label data analysis related to modern applications that involve multiple concepts [1], knowledge-mining research has provided information that is vital to achieve the distinct objectives of these applications. Such applications include conventional text categorization [2], image annotation, sentiment analysis for brand and social network service such as Twitter [3].Large numbers of features degrades the speeds of machine learning algorithms, the generality of knowledge, and the interpretability of the explored models [4]. Multi-label feature selection is considered a solution that can effectively avoid the aforementioned problems [5], [6]. Conventional multi-label feature selection methods evaluate the importance of each feature independently; therefore, the dependencies among features are ignored [2]. As a result, a compact multi-label feature subset cannot be obtained because a selected feature subset will necessarily contain redundant features, that is, features that are similar to one another [6]. To resolve this practical problem, a multi-label feature selection method must consider the feature dependencies during its feature selection process. However, these methods typically require additional computation to evaluate the feature dependencies.Recently, multi-label quadratic programming feature selection (MLQPFS) was introduced by Lim et al. [7]. It has the advantage that it concurrently considers the dependencies between the features and labels and among the features by using a quadratic function without a special search algorithm. However, although this method has this advantage, it still requires additional computational time O(N 2 ) (N is the number of features) to determine the feature dependencies.In this paper, we propose a fast multi-label feature selection method that considers the feature dependencies. To develop this method, we extended the MLQPFS method and endeavored to reduce the computational requirements involved in determining the feature dependencies by using a low-rank approximation. We decreased the time required for feature dependency determination from O(N 2 ) to O(Nk) (k is the selected number from N features and is much smaller than N) by using the MLQPFS method.

show abstract

Multilabel causal variable discovery in multisource

Wang

Lin

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

Multilabel causal feature selection, as a well-known and effective approach in dealing with high-dimensional multilabel data, is a popular topic. Amount of causal feature selection algorithms have achieved a great deal of success in classification and prediction tasks. However, the descriptive information of data is collected from different data sources in many practical applications. While few researches focus on the causal variable discovery in multisource environments due to the complex causal relationships. To address these problems, we propose a causal feature selection framework in multisource environments to solve the above problems. Firstly, we mine the causal mechanism with respect to the class attribute under the assumption that only a single data source is included. Secondly, by utilizing the concept of causal invariance in causal inference, we formulate the problem of causal feature selection with multiple data sources as a search problem for an invariant set across data sources. In addition, we give the upper and lower bounds of the causal invariant set. Finally, we design a novel multisource multilabel causal feature selection (MMCFS) algorithm. To verify the effectiveness of the proposed algorithm, we compare it with 12 feature selection methods on synthetic datasets. Experiment results show that the classification performance of MMCFS achieves highly competitive performance against other comparing algorithms.

show abstract

Feature selection for multi-label classification using multivariate mutual information

Cited by 251 publications

References 15 publications

Information Theoretic Feature Selection in Multi-label Data through Composite Likelihood

Information Theoretic Feature Selection in Multi-label Data through Composite Likelihood

Accelerating Multi-Label Feature Selection Based on Low-Rank Approximation

Multilabel causal variable discovery in multisource

Contact Info

Product

Resources

About