Statistical Instance-Based Pruning in Ensembles of Independent Classifiers

Hernández-Lobato, Daniel; Martinez-Muoz, G.; Suárez, Alberto

doi:10.1109/tpami.2008.204

Cited by 48 publications

(65 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, l}, and T k * t > T j for j = k * t , where k * t is the majority class after querying the first t classifiers. See [6] for further details.…”

Section: Statistical Instance-based Ensemble Pruningmentioning

confidence: 99%

“…Therefore, for a given T there are only (T + 1)T /2 different outcomes for (6) and T + 1 for (5). This is a fairly small number of values that can be easily precomputed and stored in an auxiliary table.…”

Section: Optimizationsmentioning

confidence: 99%

“…Thus, each term in the summation that appears in (2) can be readily computed as a product of l of these factors. In addition, to avoid numerical difficulties with the computations that involve factorials, (6) and (5) are calculated using the prime-factor decomposition of the numbers involved.…”

Section: Optimizationsmentioning

confidence: 99%

“…Therefore, for a given instance, the querying process can be halted when this probability is above a specified threshold. This dynamical instance-based ensemble pruning can be used in practice to speed-up the classification process [6,7]. For statistical IB-pruning to be effective, the probability that the final decision of the ensemble will not change when taking into account the remaining votes needs to be rapidly computed.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Statistical Instance-Based Ensemble Pruning for Multi-class Problems

Martínez-Muñoz

Hernández-Lobato

Suárez

2009

Artificial Neural Networks – ICANN 2009

View full text Add to dashboard Cite

Esta es la versión de autor de la comunicación de congreso publicada en: This is an author produced version of a paper published in:Artificial Neural Networks -ICANN 2009: 19th International Conference, Limassol, Cyprus, September 14-17, 2009 Abstract. Recent research has shown that the provisional count of votes of an ensemble of classifiers can be used to estimate the probability that the final ensemble prediction coincides with the current majority class. For a given instance, querying can be stopped when this probability is above a specified threshold. This instance-based ensemble pruning procedure can be efficiently implemented if these probabilities are pre-computed and stored in a lookup table. However, the size of the table and the cost of computing the probabilities grow very rapidly with the number of classes of the problem. In this article we introduce a number of computational optimizations that can be used to make the construction of the lookup table feasible. As a result, the application of instance-based ensemble pruning is extended to multi-class problems. Experiments in several UCI multi-class problems show that instance-based pruning speeds-up classification by a factor between 2 and 10 without any significant variation in the prediction accuracy of the ensemble.

show abstract

“…, l}, and T k * t > T j for j = k * t , where k * t is the majority class after querying the first t classifiers. See [6] for further details.…”

Section: Statistical Instance-based Ensemble Pruningmentioning

confidence: 99%

Section: Optimizationsmentioning

confidence: 99%

Section: Optimizationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Statistical Instance-Based Ensemble Pruning for Multi-class Problems

Martínez-Muñoz

Hernández-Lobato

Suárez

2009

Artificial Neural Networks – ICANN 2009

View full text Add to dashboard Cite

show abstract

“…However, larger ensembles have larger storage needs and longer times of prediction. To alleviate these shortcomings, different ensemble pruning methods can be used [4][5][6][7][8][9][10][11][12][13]. The goal of these methods is to reduce the memory requirements and to speed-up the classification process while maintaining or, if possible, improving the level of accuracy of the original ensemble.…”

Section: Introductionmentioning

confidence: 99%

A Double Pruning Algorithm for Classification Ensembles

Soto

Martínez-Muñoz

Hernández-Lobato

et al. 2010

Multiple Classifier Systems

View full text Add to dashboard Cite

Esta es la versión de autor de la comunicación de congreso publicada en: This is an author produced version of a paper published in: Abstract. This article introduces a double pruning algorithm that can be used to reduce the storage requirements, speed-up the classification process and improve the performance of parallel ensembles. A key element in the design of the algorithm is the estimation of the class label that the ensemble assigns to a given test instance by polling only a fraction of its classifiers. Instead of applying this form of dynamical (instance-based) pruning to the original ensemble, we propose to apply it to a subset of classifiers selected using standard ensemble pruning techniques. The pruned subensemble is built by first modifying the order in which classifiers are aggregated in the ensemble and then selecting the first classifiers in the ordered sequence. Experiments in benchmark problems illustrate the improvements that can be obtained with this technique. Specifically, using a bagging ensemble of 101 CART trees as a starting point, only the 21 trees of the pruned ordered ensemble need to stored in memory. Depending on the classification task, on average, only 5 to 12 of these 21 classifiers are queried to compute the predictions. The generalization performance achieved by this double pruning algorithm is similar to pruned ordered bagging and significantly better than standard bagging.

show abstract

Multilayer hybrid strategy for phishing email zero‐day filtering

Chowdhury

Abawajy

Kelarev

et al. 2016

Concurrency and Computation

View full text Add to dashboard Cite

Summary The cyber security threats from phishing emails have been growing buoyed by the capacity of their distributors to fine‐tune their trickery and defeat previously known filtering techniques. The detection of novel phishing emails that had not appeared previously, also known as zero‐day phishing emails, remains a particular challenge. This paper proposes a multilayer hybrid strategy (MHS) for zero‐day filtering of phishing emails that appear during a separate time span by using training data collected previously during another time span. This strategy creates a large ensemble of classifiers and then applies a novel method for pruning the ensemble. The majority of known pruning algorithms belong to the following three categories: ranking based, clustering based, and optimization‐based pruning. This paper introduces and investigates a multilayer hybrid pruning. Its application in MHS combines all three approaches in one scheme: ranking, clustering, and optimization. Furthermore, we carry out thorough empirical study of the performance of the MHS for the filtering of phishing emails. Our empirical study compares the performance of MHS strategy with other machine learning classifiers. The results of our empirical study demonstrate that MHS achieved the best outcomes and multilayer hybrid pruning performed better than other pruning techniques. Copyright © 2016 John Wiley & Sons, Ltd.

show abstract

Statistical Instance-Based Pruning in Ensembles of Independent Classifiers

Cited by 48 publications

References 13 publications

Statistical Instance-Based Ensemble Pruning for Multi-class Problems

Statistical Instance-Based Ensemble Pruning for Multi-class Problems

A Double Pruning Algorithm for Classification Ensembles

Multilayer hybrid strategy for phishing email zero‐day filtering

Contact Info

Product

Resources

About