Pruning in ordered bagging ensembles

Martínez-Muñoz, Gonzalo; Suárez, Alberto

doi:10.1145/1143844.1143921

Cited by 130 publications

(86 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…After obtaining the component learners, most ensemble algorithms combine all of them to build an ensemble, however, it has been shown that it is better to ensemble some instead of all of them [24,23,17].…”

Section: Related Workmentioning

confidence: 99%

“…Martínez-Muñoz and Suárez [17] proposed a heuristic method, where the component learners obtained from bagging are reordered, and a part of the top-ranked ones are included in the final ensemble. The experimental result also shows that selective ensemble may improve ensemble's performance while reducing its size.…”

Section: Related Workmentioning

confidence: 99%

“…However, some researchers [24,23,17] show that through selective ensemble, i.e., ensembling some instead of all the available component learners, a better ensemble can be generated. The selection, however, is not easy and thus many heuristics have been used.…”

Section: Introductionmentioning

confidence: 99%

“…The selection, however, is not easy and thus many heuristics have been used. For example, for the selection, Zhou et al [24,23] used a genetic algorithm; Castro et al [7] employed artificial immune algorithm; Coyle and Smith [9] utilized case similarity; Martínez-Muñoz and Suárez [17] proposed to order the component learners and then select the first ones to use.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Selective Ensemble under Regularization Framework

Zhou

2009

Multiple Classifier Systems

View full text Add to dashboard Cite

Abstract. An ensemble is generated by training multiple component learners for a same task and then combining them for predictions. It is known that when lots of trained learners are available, it is better to ensemble some instead of all of them. The selection, however, is generally difficult and heuristics are often used. In this paper, we investigate the problem under the regularization framework, and propose a regularized selective ensemble algorithm RSE. In RSE, the selection is reduced to a quadratic programming problem, which has a sparse solution and can be solved efficiently. Since it naturally fits the semi-supervised learning setting, RSE can also exploit unlabeled data to improve the performance. Experimental results show that RSE can generate ensembles with small size but strong generalization ability.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Selective Ensemble under Regularization Framework

Zhou

2009

Multiple Classifier Systems

View full text Add to dashboard Cite

show abstract

“…However, the ensemble pruning is a difficult problem whose solution is commonly computationally expensive. Pruning an ensemble with n models requires searching in the space of the 2 n − 1 non-empty solutions to minimize a cost function correlated with the generalization error [18].…”

Section: Introductionmentioning

confidence: 99%

Comparison of a genetic algorithm and simulated annealing for automatic neural network ensemble development

2013

View full text Add to dashboard Cite

In the last decades ensemble learning has established itself as a valuable strategy within the computational intelligence modeling and machine learning community. Ensemble learning is a paradigm where multiple models combine in some way their decisions, or their learning algorithms, or different data to improve the prediction performance. Ensemble learning aims at improving the generalization ability and the reliability of the system. Key factors of ensemble systems are diversity, training and combining ensemble members to improve the ensemble system performance. Since there is no unified procedure to address all these issues, this work proposes and compares Genetic Algorithm and Simulated Annealing based approaches for the automatic development of Neural Network Ensembles for regression problems. The main contribution of this work is the development of optimization techniques that select the best subset of models to be aggregated taking into account all the key factors of ensemble systems (e.g., diversity, training ensemble members and combination strategy). Experiments on two well-known data sets are reported to evaluate the effectiveness of the proposed methodologies. Results show that these outperform other approaches including Simple Bagging, Negative Correlation Learning (NCL), AdaBoost and GASEN in terms of generalization ability.

show abstract