Selective Ensemble of Classifier Chains

Li, Nan; Zhou, Zhi‐Hua

doi:10.1007/978-3-642-38067-9_13

Cited by 21 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All these method have in common with the original CC that they can only effectively exploit one direction of a dependency between two labels. Li and Zhou (2013) propose to filter the CCs of an ensemble of CCs (Read et al 2011) in order to maximize F1. Unfortunately, no post-analysis is performed of the useful pairing directions.…”

Section: Label Dependencies and Stackingmentioning

confidence: 99%

Learning rules for multi-label classification: a stacking and a separate-and-conquer approach

Mencía

Janssen

2016

Mach Learn

View full text Add to dashboard Cite

Dependencies between the labels are commonly regarded as the crucial issue in multi-label classification. Rules provide a natural way for symbolically describing such relationships. For instance, rules with label tests in the body allow for representing directed dependencies like implications, subsumptions, or exclusions. Moreover, rules naturally allow to jointly capture both local and global label dependencies. In this paper, we introduce two approaches for learning such label-dependent rules. Our first solution is a bootstrapped stacking approach which can be built on top of a conventional rule learning algorithm. For this, we learn for each label a separate ruleset, but we include the remaining labels as additional attributes in the training instances. The second approach goes one step further by adapting the commonly used separate-and-conquer algorithm for learning multi-label rules. The main idea is to re-include the covered examples with the predicted labels so that this information can be used for learning subsequent rules. Both approaches allow for making label dependencies explicit in the rules. In addition, the usage of standard rule learning techniques targeted at producing accurate predictions ensures that the found rules are useful for actual classification. Our experiments show (a) that the discovered dependencies contribute to the understanding and improve the analysis of multi-label datasets, and (b) that the found multi-label rules are crucial for the predictive performance as our proposed approaches beat the baseline using conventional rules.

show abstract

Section: Label Dependencies and Stackingmentioning

confidence: 99%

Learning rules for multi-label classification: a stacking and a separate-and-conquer approach

Mencía

Janssen

2016

Mach Learn

View full text Add to dashboard Cite

show abstract

“…In traditional supervised learning, a number of methods have been developed based on different techniques, such as genetic algorithm [35], semi-definite programming [33], clustering [9], 1 -norm regularized sparse optimization [14]. In [15], in order to reduce the size of ECC, Li and Zhou proposed SECC (i.e., selective ensemble of classifier chains), which to our best knowledge is the first work on selective ensemble in the multi-label setting.…”

Section: Related Workmentioning

confidence: 99%

“…Note that the ensemble size of ECC100 is 100, and the Random strategy generates ensembles of the same size of MUSE. [12,17,15]. Moreover, compared with previous work [15], the MUSE approach is more general, for instance, it can optimize a large variety of performance measures while SECC considers only F1-score.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Multi-label Selective Ensemble

Li¹,

Jiang²,

Zhou³

2015

Multiple Classifier Systems

Self Cite

View full text Add to dashboard Cite

Multi-label selective ensemble deals with the problem of reducing the size of multi-label ensembles whilst keeping or improving the performance. In practice, it is of important value, since the generated ensembles are usually unnecessarily large, which leads to extra high computational and storage cost. However, it is more challenging than traditional selective ensemble, because real-world applications often employ different performance measures to evaluate the quality of multi-label predictions, depending on user requirements. In this paper, we propose the MUSE approach to tackle this problem. Specifically, by directly considering the concerned performance measure, we develop a convex optimization formulation and provide an efficient stochastic optimization solution for a large variety of multi-label performance measures. Experiments show that MUSE is able to obtain smaller multi-label ensembles, whilst achieving better or at least comparable performance in terms of the concerned performance measure.

show abstract

“…Previous studies (Vinyals, Bengio, and Kudlur 2016;Yang et al 2018a) show that ordering has a significant impact on the performance. This issue also appears in the PCC, It is addressed by ensemble averaging (Read et al 2011;Cheng, Hüllermeier, and Dembczynski 2010), ensemble pruning (Li and Zhou 2013), pre-analysis of the label dependencies by Bayes nets (Sucar et al 2014) and integrating beam search with training to determine a suitable tag ordering (Kumar et al 2013). However, these approaches rely on training multiple models to ensemble or determine a proper order, which is computationally expensive.…”

Section: Introductionmentioning

confidence: 99%

Order-Free Learning Alleviating Exposure Bias in Multi-Label Classification

Tsai

Lee

2020

AAAI

View full text Add to dashboard Cite

Multi-label classification (MLC) assigns multiple labels to each sample. Prior studies show that MLC can be transformed to a sequence prediction problem with a recurrent neural network (RNN) decoder to model the label dependency. However, training a RNN decoder requires a predefined order of labels, which is not directly available in the MLC specification. Besides, RNN thus trained tends to overfit the label combinations in the training set and have difficulty generating unseen label sequences. In this paper, we propose a new framework for MLC which does not rely on a predefined label order and thus alleviates exposure bias. The experimental results on three multi-label classification benchmark datasets show that our method outperforms competitive baselines by a large margin. We also find the proposed approach has a higher probability of generating label combinations not seen during training than the baseline models. The result shows that the proposed approach has better generalization capability.

show abstract

Selective Ensemble of Classifier Chains

Cited by 21 publications

References 24 publications

Learning rules for multi-label classification: a stacking and a separate-and-conquer approach

Learning rules for multi-label classification: a stacking and a separate-and-conquer approach

Multi-label Selective Ensemble

Order-Free Learning Alleviating Exposure Bias in Multi-Label Classification

Contact Info

Product

Resources

About