DIVACE: Diverse and Accurate Ensemble Learning Algorithm

Chandra, Arjun; Yao, Xin

doi:10.1007/978-3-540-28651-6_91

Cited by 57 publications

(52 citation statements)

References 11 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there is a tradeoff between accuracy and diversity and it is essential that the ensemble members are highly diverse and sufficiently accurate [52], [53]. Previously, the diversity of the ensemble members has been promoted through the use of different data, different learning algorithms or different learning models [54].…”

Section: Diverse Ensemble Generationmentioning

confidence: 99%

See 1 more Smart Citation

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Jin

Sendhoff

2008

IEEE Trans. Syst., Man, Cybern. C

324

View full text Add to dashboard Cite

Abstract-Machine learning is inherently a multiobjective task. Traditionally, however, either only one of the objectives is adopted as the cost function or multiple objectives are aggregated to a scalar cost function. This can be mainly attributed to the fact that most conventional learning algorithms can only deal with a scalar cost function. Over the last decade, efforts on solving machine learning problems using the Pareto-based multiobjective optimization methodology have gained increasing impetus, particularly due to the great success of multiobjective optimization using evolutionary algorithms and other population-based stochastic search methods. It has been shown that Pareto-based multiobjective learning approaches are more powerful compared to learning algorithms with a scalar cost function in addressing various topics of machine learning, such as clustering, feature selection, improvement of generalization ability, knowledge extraction, and ensemble generation. One common benefit of the different multiobjective learning approaches is that a deeper insight into the learning problem can be gained by analyzing the Pareto front composed of multiple Pareto-optimal solutions. This paper provides an overview of the existing research on multiobjective machine learning, focusing on supervised learning. In addition, a number of case studies are provided to illustrate the major benefits of the Pareto-based approach to machine learning, e.g., how to identify interpretable models and models that can generalize on unseen data from the obtained Pareto-optimal solutions. Three approaches to Pareto-based multiobjective ensemble generation are compared and discussed in detail. Finally, potentially interesting topics in multiobjective machine learning are suggested.

show abstract

Section: Diverse Ensemble Generationmentioning

confidence: 99%

“…Traditionally, the approximation error and the output correlation between the ensemble members are summed up to a scalar objective function [8], [55]. In [52], the Pareto-based approach is adopted to generate diverse and accurate ensembles, where the following two objectives are minimized,…”

Section: Diverse Ensemble Generationmentioning

confidence: 99%

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Jin

Sendhoff

2008

IEEE Trans. Syst., Man, Cybern. C

324

View full text Add to dashboard Cite

show abstract

“…There may not be a large diversity in classification results by non-dominated rule sets obtained by multiobjective fuzzy rule selection. A promising research direction is to combine a diversity-maintenance mechanism of rule sets (or their classification results) into evolutionary multiobjective search as in [4,5] used for the design of ensemble neural network classifiers. The use of fuzzy geneticsbased machine learning (e.g., [26,35]) instead of fuzzy rule selection is also a promising research direction because no prescreening stage of candidate fuzzy rules is needed.…”

Section: Discussionmentioning

confidence: 99%

“…An ensemble classifier was constructed by combining non-dominated neural networks with respect to the two objectives. On the other hand, Chandra and Yao [4,5] used a different formulation in order to increase the diversity of neural networks in a more direct manner. They formulated a two-objective optimization problem using an accuracy measure and a diversity measure.…”

Section: Introductionmentioning

confidence: 99%

Fuzzy Ensemble Design through Multi-Objective Fuzzy Rule Selection

Ishibuchi

Nojima

Multi-Objective Machine Learning

View full text Add to dashboard Cite

Summary.The main advantage of evolutionary multi-objective optimization (EMO) over classical approaches is that a variety of non-dominated solutions with a wide range of objective values can be simultaneously obtained by a single run of an EMO algorithm. In this chapter, we show how this advantage can be utilized in the design of fuzzy ensemble classifiers. First we explain three objectives in multi-objective formulations of fuzzy rule selection. One is accuracy maximization and the others are complexity minimization. Next we demonstrate that a number of non-dominated rule sets (i.e., fuzzy classifiers) are obtained along the accuracy-complexity tradeoff surface from multi-objective fuzzy rule selection problems. Then we examine the effect of combining multiple non-dominated fuzzy classifiers into a single ensemble classifier. Experimental results clearly show that the combination into ensemble classifiers improves the classification ability of individual fuzzy classifiers for some data sets. IntroductionA promising approach to the design of reliable classifiers is to combine multiple classifiers into a single one [2,10]. Several methods have been proposed for generating multiple classifiers such as bagging [3] and boosting [12]. The point in classifier aggregation is to generate an ensemble of classifiers with high diversity. Ideally the classification errors by each classifier should be uncorrelated. In this chapter, we examine the use of evolutionary multiobjective optimization (EMO) algorithms for generating an ensemble of fuzzy rule-based classifiers with high diversity.The main advantage of evolutionary multiobjective optimization (EMO) over classical approaches is that many non-dominated solutions can be simultaneously obtained by a single run of an EMO algorithm. When an EMO algorithm is used in the design of an ensemble classifier, a number of nondominated classifiers with high diversity can be obtained by its single run. This advantage has already been utilized in some studies on the design of neural

show abstract

“…Second, it is possible to simultaneously generate multiple learning models that account for different learning goals, e.g., accuracy and complexity [20], [22], multiple error measures [12], interpretability and accuracy [24]. Third, the multiple learning models produced using multi-objective optimization are well suited for constructing learning ensembles [2], [10], [22]. And finally, more information can be gained by analyzing the Pareto front obtained in multi-objective machine learning.…”

Section: Introductionmentioning

confidence: 99%

Alleviating Catastrophic Forgetting via Multi-Objective Learning

Jin¹,

Sendhoff²

The 2006 IEEE International Joint Conference on Neural Network Proceedings

View full text Add to dashboard Cite

Abstract-Handling catastrophic forgetting is an interesting and challenging topic in modeling the memory mechanisms of the human brain using machine learning models. From a more general point of view, catastrophic forgetting reflects the stability-plasticity dilemma, which is one of the several dilemmas to be addressed in learning systems: to retain the stored memory while learning new information. Different to the existing approaches, we introduce a Pareto-optimality based multi-objective learning framework for alleviating catastrophic learning. Compared to the single-objective learning methods, multi-objective evolutionary learning with the help of pseudorehearsal is shown to be more promising in dealing with the stability-plasticity dilemma.

show abstract

DIVACE: Diverse and Accurate Ensemble Learning Algorithm

Cited by 57 publications

References 11 publications

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Fuzzy Ensemble Design through Multi-Objective Fuzzy Rule Selection

Alleviating Catastrophic Forgetting via Multi-Objective Learning

Contact Info

Product

Resources

About