Embedded local feature selection within mixture of experts

Peralta, Billy; Soto, Álvaro

doi:10.1016/j.ins.2014.01.008

Cited by 51 publications

(25 citation statements)

References 31 publications

(10 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We studied the two-class classification problem in these experiments and used the logistic regression models as the local expert models. The proposed trace norm regularized MOE model is compared with the L 1 norm regularized MOE model [22], support vector machines (SVM), linear logistic regression with L 1 norm regularization, SVM ensemble with bagging, and AdaBoost using decision trees as weak classifiers. The parameters of these methods are selected with 3-fold cross-validation using the grid search.…”

Section: Methodsmentioning

confidence: 99%

“…[20]. Recently, many novel MOE methods are proposed to handle high dimensional data [21][22][23]. In this paper, a new trace norm regularized MOE model is proposed.…”

Section: Introductionmentioning

confidence: 99%

“…The trace norm regularized MOE model can uncover the shared underlying characteristics of training input. Different from the previous study on the MOE model which often aims to select a set of sparse features [22], the trace norm regularized MOE model gains a small set of underlying characteristics which can be represented by the linear combination of original features. Moreover, the trace norm regularization allows us to work in the kernel space and handle the high dimensional (or infinite) features.…”

Section: Introductionmentioning

confidence: 99%

“…Moreover, the trace norm regularization allows us to work in the kernel space and handle the high dimensional (or infinite) features. At this point, the trace norm regularized MOE model is more flexible than the MOE with sparse feature selection [22]. This paper is organized as follows.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Multi-task feature learning by using trace norm regularization

Zhang

et al. 2017

Open Physics

View full text Add to dashboard Cite

Abstract:Multi-task learning can extract the correlation of multiple related machine learning problems to improve performance. This paper considers applying the multi-task learning method to learn a single task. We propose a new learning approach, which employs the mixture of expert model to divide a learning task into several related subtasks, and then uses the trace norm regularization to extract common feature representation of these sub-tasks. A nonlinear extension of this approach by using kernel is also provided. Experiments conducted on both simulated and real data sets demonstrate the advantage of the proposed approach.

show abstract

Section: Methodsmentioning

confidence: 99%

“…[20]. Recently, many novel MOE methods are proposed to handle high dimensional data [21][22][23]. In this paper, a new trace norm regularized MOE model is proposed.…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-task feature learning by using trace norm regularization

Zhang

et al. 2017

Open Physics

View full text Add to dashboard Cite

show abstract

“…In general, feature selection algorithms are classified into two categories: those based on the filter model [8] and those based on the wrapper model [27]. A number of feature selection methods based on the wrapper model for classification [30,38] and clustering [52] have been proposed in the literature. In supervised learning, one can generally use the wrapper model to construct a classifier and use the criterion (e.g., accuracy) to observe how the features can positively predict class labels via this classifier [10,35,36,40].…”

Section: Related Workmentioning

confidence: 99%

Feature selection for clustering using instance-based learning by exploring the nearest and farthest neighbors

Chen

2015

Information Sciences

View full text Add to dashboard Cite

Practical and theoretical aspects of mixture‐of‐experts modeling: An overview

Nguyen

Chamroukhi

2018

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Mixture-of-experts (MoE) models are a powerful paradigm for modeling data arising from complex data generating processes (DGPs). In this article, we demonstrate how different MoE models can be constructed to approximate the underlying DGPs of arbitrary types of data. Due to the probabilistic nature of MoE models, we propose the maximum quasi-likelihood (MQL) approach as a method for estimating MoE model parameters from data, and we provide conditions under which MQL estimators are consistent and asymptotically normal. The blockwise minorizationmaximization (blockwise-MM) algorithm framework is proposed as an all-purpose method for constructing algorithms for obtaining MQL estimators. An example derivation of a blockwise-MM algorithm is provided. We then present a method for constructing information criteria for estimating the number of components in MoE models and provide justification for the classic Bayesian information criterion (BIC). We explain how MoE models can be used to conduct classification, clustering, and regression and illustrate these applications via two worked examples.

show abstract

Embedded local feature selection within mixture of experts

Cited by 51 publications

References 31 publications

Multi-task feature learning by using trace norm regularization

Multi-task feature learning by using trace norm regularization

Feature selection for clustering using instance-based learning by exploring the nearest and farthest neighbors

Practical and theoretical aspects of mixture‐of‐experts modeling: An overview

Contact Info

Product

Resources

About