Inverse regression approach to robust nonlinear high-to-low dimensional mapping

Abstract

International audienceThe goal of this paper is to address the issue of nonlinear regression with outliers, possibly in high dimension, without specifying the form of the link function and under a parametric approach. Nonlinearity is handled via an underlying mixture of affine regressions. Each regression is encoded in a joint multivariate Student distribution on the responses and covariates. This joint modeling allows the use of an inverse regression strategy to handle the high dimensionality of the data, while the heavy tail of the Student distribution limits the contamination by outlying data. The possibility to add a number of latent variables similar to factors to the model further reduces its sensitivity to noise or model misspecification. The mixture model setting has the advantage of providing a natural inference procedure using an EM algorithm. The tractability and flexibility of the algorithm are illustrated in simulations and real high-dimensional data with good performance that compares favorably with other existing methods

Prediction with High Dimensional Regression Via Hierarchically Structured Gaussian Mixtures and Latent Variables

“…The goal is to use the spectra to predict the level of sucrose (L t = 1). We follow the step that was described in Perthame et al (2018) and decompose the spectra on a spline basis with (D = 134) to make D ≈ N. This data set is known for outliers; the realization of X and T is given in Fig. 4.…”

“…This method is referred to as GLLiM-outlier. In addition, we consider SLLiM in Perthame et al (2018) provided by the R package xLLiM (Perthame et al, 2017). SLLiM is a counterpart of GLLiM that accommodates abnormal samples by using Student t-distributions.…”

Approximation results regarding the multiple-output Gaussian gated mixture of linear experts model

“…A class of related gating functions to (5) are the student-t gates. This type of gating has been explored in [16,21,22]. Multivariate probit gates have also been considered in [23].…”

Practical and theoretical aspects of mixture‐of‐experts modeling: An overview

“…Although it is possible to utilize any set of functions that meet the restrictions Gate z ( x ; γ ) > 0, and , there are few alternatives to and that are considered in the literature. Some of these considered alternatives include the exponential family gating functions of Xu et al () and the Student‐ t gating functions of Perthame, Forbes, Olivier, and Deleforge ().Remark Aside from the simple MoE models that can be characterized via the simple architecture of Figure , there are more intricate constructions that are possible for the modeling of complex data. Examples of extensions to the MoE modeling framework include the Mixed‐effects MoE models of Ng and McLachlan () and Ng and McLachlan (), and the hierarchical MoE models of Jordan and Jacobs () and Jordan and Jacobs () that can be used to fit highly heterogeneous and nonlinear data.…”