Modelling Conditional Densities Using Finite Smooth Mixtures

Abstract

ABSTRACT. Smooth mixtures, i.e. mixture models with covariate-dependent mixing weights, are very useful flexible models for conditional densities. Previous work shows that using too simple mixture components for modeling heteroscedastic and/or heavy tailed data can give a poor fit, even with a large number of components. This paper explores how well a smooth mixture of symmetric components can capture skewed data. Simulations and applications on real data show that including covariate-dependent skewness in the components can lead to substantially improved performance on skewed data, often using a much smaller number of components. Furthermore, variable selection is effective in removing unnecessary covariates in the skewness, which means that there is little loss in allowing for skewness in the components when the data are actually symmetric. We also introduce smooth mixtures of gamma and log-normal components to model positively-valued response variables.

Regression Density Estimation With Variational Methods and Stochastic Approximation

“…They also observed that with MHR models, fewer mixture components are required, which makes the estimation and interpretation of mixture models easier. In an analysis of the benchmark LIDAR dataset, Li, Villani, and Kohn (2010a) showed that a model with homoscedastic thin-plate components requires three components to achieve approximately the same performance as an MHR model with a single thin-plate component, providing further evidence that MHR models help in reducing the number of mixture components required. Moreover, Villani, Kohn, and Giordani (2009) demonstrated that the fit of an MHR model to homoscedastic data is comparable with that of a model with homoscedastic components.…”

Semiparametric finite mixture of regression models with Bayesian P-splines

“…In particular, a multinomial logistic regression structure is commonly chosen to link the component weights to the regressors. Applications of FMRC models are described in the statistical, econometric and machine learning literature (see, for example, Weigend and Shi 2000;Lu 2006; Gormley and Murphy 2008;Villani et al 2009;Lê Cao et al 2010;Li et al 2010Li et al , 2011Frühwirth-Schnatter et al 2012;Gormley and Frühwirth-Schnatter 2019;. It is worth mentioning that some of these applications consider multivariate regressors and/or multivariate dependent variables.…”

Handbook of Mixture Analysis

“…The mixture of experts nomenclature (ME) has its origins in the machine-learning literature (Jacobs et al , 1991), but mixtures of experts models appear in many different guises, including switching regression models (Quandt, 1972), concomitant variable latentclass models (Dayton & Macready, 1988), latent class regression models (DeSarbo & Cron, 1988), and mixed models (Wang et al , 1996). Li et al (2011) discuss finite smooth mixtures, a special case of ME modelling. McLachlan & Peel (2000) and Frühwirth-Schnatter (2006) provide background to a range of mixtures of experts models; Masoudnia & Ebrahimpour (2014) survey the ME literature from a machine learning perspective.…”