Selective Ensemble under Regularization Framework

Li, Nan; Zhou, Zhi‐Hua

doi:10.1007/978-3-642-02326-2_30

Cited by 44 publications

(36 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ensemble pruning or ensemble selection) [30,Chapter 6] is an active research topic, and many technologies has been used to build selective ensembles, such as genetic algorithm [31], semi-definite programming [29], clustering [9], sparse optimization [13]. Obviously, ECC is an ensemble of multi-label classifiers, making the current work essentially different.…”

Section: Related Workmentioning

confidence: 99%

Selective Ensemble of Classifier Chains

Zhou

2013

Multiple Classifier Systems

Self Cite

View full text Add to dashboard Cite

Abstract. In multi-label learning, the relationship among labels is well accepted to be important, and various methods have been proposed to exploit label relationships. Amongst them, ensemble of classifier chains (ECC) which builds multiple chaining classifiers by random label orders has drawn much attention. However, the ensembles generated by ECC are often unnecessarily large, leading to extra high computational and storage cost. To tackle this issue, in this paper, we propose selective ensemble of classifier chains (SECC) which tries to select a subset of classifier chains to composite the ensemble whilst keeping or improving the performance. More precisely, we focus on the performance measure F1-score, and formulate this problem as a convex optimization problem which can be efficiently solved by the stochastic gradient descend method. Experiments show that, compared with ECC, SECC is able to obtain much smaller ensembles while achieving better or at least comparable performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Selective Ensemble of Classifier Chains

Zhou

2013

Multiple Classifier Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…The first group of methods use global search to directly select the optimal or near-optimal classifier subset. In the literature, many techniques have been used, such as genetic algorithm [35], semi-definite programming [31], clustering [12,17], sparse optimization with sparsity-inducing prior [7] or 1 -norm constraint [18], etc. In practice, this kind of methods can achieve good performance, but their computational costs are usually quite large.…”

Section: Related Workmentioning

confidence: 99%

Diversity Regularized Ensemble Pruning

Zhou

2012

Lecture Notes in Computer Science

Self Cite

100

View full text Add to dashboard Cite

Abstract. Diversity among individual classifiers is recognized to play a key role in ensemble, however, few theoretical properties are known for classification. In this paper, by focusing on the popular ensemble pruning setting (i.e., combining classifier by voting and measuring diversity in pairwise manner), we present a theoretical study on the effect of diversity on the generalization performance of voting in the PAC-learning framework. It is disclosed that the diversity is closely-related to the hypothesis space complexity, and encouraging diversity can be regarded to apply regularization on ensemble methods. Guided by this analysis, we apply explicit diversity regularization to ensemble pruning, and propose the Diversity Regularized Ensemble Pruning (DREP) method. Experimental results show the effectiveness of DREP.

show abstract

“…In 2002, Zhou et al proved that it may be better to combine many instead of all of the learners, and the proposed GASEN algorithm can generate ensembles with smaller sizes but stronger generalization ability [14]. In 2009, Nan Li and Zhi-Hua Zhou proposed selective ensemble method under regularization framework [15].…”

Section: Related Workmentioning

confidence: 98%

Trees Weighting Random Forest Method for Classifying High-Dimensional Noisy Data

Wang

Ding

et al. 2010

2010 IEEE 7th International Conference on E-Business Engineering

View full text Add to dashboard Cite

Random forest is an excellent ensemble learning method, which is composed of multiple decision trees grown on random input samples and splitting nodes on a random subset of features. Due to its good classification and generalization ability, random forest has achieved success in various domains. However, random forest will generate many noisy trees when it learns from the data set that has high dimension with many noise features. These noisy trees will affect the classification accuracy, and even make a wrong decision for new instances. In this paper, we present a new approach to solve this problem through weighting the trees according to their classification ability, which is named Trees Weighting Random Forest (TWRF). Here, Out-Of-Bag, which is the training data subset generated by Bagging and not involved in building decision tree, is used to evaluate the tree. For simplicity, we choose the accuracy as the index that notes tree's classification ability and set it as the tree's weight. Experiments show that TWRF has better performance than the original random forest and other traditional methods, such as C45, Naïve Bayes and so on.

show abstract

Selective Ensemble under Regularization Framework

Cited by 44 publications

References 13 publications

Selective Ensemble of Classifier Chains

Selective Ensemble of Classifier Chains

Diversity Regularized Ensemble Pruning

Trees Weighting Random Forest Method for Classifying High-Dimensional Noisy Data

Contact Info

Product

Resources

About