Mixture of Experts Modelling with Social Science Applications

Gormley, Isobel Claire

doi:10.1002/9781119995678.ch5

Cited by 14 publications

(16 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, T , after the burn-in. AICM has already been applied successfully in the mixture modelling context, for example by Erosheva et al (2007), Gormley and Murphy (2010), Gormley and Murphy (2011), and Mollica and Tardella (2017).…”

Section: Posterior Inference and Model Selectionmentioning

confidence: 99%

Flexible Bayesian modelling of concomitant covariate effects in mixture models

Berrettini,

Galimberti,

Ranciati

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Mixture models provide a useful tool to account for unobserved heterogeneity and are at the basis of many model-based clustering methods. In order to gain additional flexibility, some model parameters can be expressed as functions of concomitant covariates. In particular, component weights of the mixture can be linked to the covariates through a multinomial logistic regression model, where each component weight is a function of the linear predictor involving one or more covariates. The proposed contribution extends this approach by replacing the linear predictor, used for the component weights, with an additive structure, where each term is a smooth function of the covariates considered. An estimation procedure within the Bayesian paradigm is suggested. In particular, a data augmentation scheme based on differenced random utility models is exploited, and smoothness of the covariate effects is controlled by suitable choices for the prior distributions of the spline coefficients. The performance of the proposed methodology is investigated via simulation experiments, and an application to an original dataset about UK parliamentary votes on Brexit is discussed.

show abstract

Section: Posterior Inference and Model Selectionmentioning

confidence: 99%

Flexible Bayesian modelling of concomitant covariate effects in mixture models

Berrettini,

Galimberti,

Ranciati

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…It's possible that some, none, or all model parameters depend on the covariates. This leads to the four special cases of the Gaussian MoE model shown in Figure 1, with the following interpretations, due to Gormley & Murphy (2011):…”

Section: The Moe Family Of Modelsmentioning

confidence: 99%

Gaussian parsimonious clustering models with covariates and a noise component

Murphy

2019

Adv Data Anal Classif

Self Cite

View full text Add to dashboard Cite

We consider model-based clustering methods for continuous, correlated data that account for external information available in the presence of mixed-type fixed covariates by proposing the MoEClust suite of models. These allow covariates influence the component weights and/or component densities by modelling the parameters of the mixture as functions of the covariates. A familiar range of constrained eigen-decomposition parameterisations of the component covariance matrices are also accommodated. This paper thus addresses the equivalent aims of including covariates in Gaussian Parsimonious Clustering Models and incorporating parsimonious covariance structures into the Gaussian mixture of experts framework. The MoEClust models demonstrate significant improvement from both perspectives in applications to univariate and multivariate data sets.Keywords: Model-based clustering, mixtures of experts, EM algorithm, parsimony, multivariate response, covariates arXiv:1711.05632v2 [stat.ME] 10 Dec 2018 cluster membership indicator vector, where z ig = 1 if observation i belongs to cluster g and z ig = 0 otherwise, the first approach assumes that z i affects the distribution of x i . In probabilistic terms, this means to replace the actual group-specific conditional dis-The name 'cluster-weighted model' (CWM) is frequently given to this approach, e.g. Dang et al. (2017) and Ingrassia et al. (2015); the latter provides a recent extension allowing for mixed-type covariates. The use of the alternative term 'mixtures of regressions with random covariates' to describe CWMs (e.g. Hennig (2000)), provides opportunity to clarify that the remainder of this paper focuses on the second approach, with fixed covariates affecting cluster membership via f (yThis is achieved using the mixture of experts (MoE) paradigm (Jacobs et al., 1991), in which the parameters of the mixture are modelled as functions of fixed, potentially mixed-type covariates. However, for multivariate, continuous, correlated responses, a unifying framework combining the special cases of the Gaussian MoE model with the flexibility afforded by the various parsimonious covariance parameterisations within the Gaussian Parsimonious Clustering Models (GPCM) family of finite mixture models (Banfield & Raftery, 1993;Celeux & Govaert, 1995) has to date been lacking. Indeed, the main contribution of this paper is in addressing the aim of incorporating covariates into the GPCM family and the equivalent aim of incorporating GPCM covariance constraints into the Gaussian MoE framework, by proposing the MoEClust family of models: the name comes from the interest in employing MoE models chiefly for clustering purposes. From both perspectives, MoEClust models demonstrate significant improvement in applications to univariate and multivariate response data.A software implementation for the full suite of MoEClust models is provided by the associated R package MoEClust (Murphy & Murphy, 2018), which is available from www.r-project.org (R Core Team, 2018), with which all results were obtaine...

show abstract

“…In addition, ME has been used in time series analysis, such as speech recognition [23], financial forecasting [33] and dynamic control systems [17, 32]. Recently, ME was used in social network analysis, in which various social behavior patterns are modeled through a mixture [12]. …”

Section: Preliminarymentioning

confidence: 99%

A Generalized Mixture Framework for Multi-label Classification

Hong

Batal

Hauskrecht

2015

Proceedings of the 2015 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

We develop a novel probabilistic ensemble framework for multi-label classification that is based on the mixtures-of-experts architecture. In this framework, we combine multi-label classification models in the classifier chains family that decompose the class posterior distribution P(Y1, …, Yd|X) using a product of posterior distributions over components of the output space. Our approach captures different input–output and output–output relations that tend to change across data. As a result, we can recover a rich set of dependency relations among inputs and outputs that a single multi-label classification model cannot capture due to its modeling simplifications. We develop and present algorithms for learning the mixtures-of-experts models from data and for performing multi-label predictions on unseen data instances. Experiments on multiple benchmark datasets demonstrate that our approach achieves highly competitive results and outperforms the existing state-of-the-art multi-label classification methods.

show abstract

Mixture of Experts Modelling with Social Science Applications

Abstract: The UCD community has made this article openly available. Please share how this access benefits you. Your story matters! (@ucd_oa) Some rights reserved. For more information, please see the item record link above.

Cited by 14 publications

References 62 publications

Flexible Bayesian modelling of concomitant covariate effects in mixture models

Flexible Bayesian modelling of concomitant covariate effects in mixture models

Gaussian parsimonious clustering models with covariates and a noise component

A Generalized Mixture Framework for Multi-label Classification

Contact Info

Product

Resources

About