Model Selection Criteria on Beta Regression for Machine Learning

Abstract

Beta regression models are a class of supervised learning tools for regression problems with univariate and limited response. Current fitting procedures for beta regression require variable selection based on (potentially problematic) information criteria. We propose model selection criteria that take into account the leverage, residuals, and influence of the observations, both to systematic linear and nonlinear components. To that end, we propose a Predictive Residual Sum of Squares (PRESS)-like machine learning tool and a prediction coefficient, namely P 2 statistic, as a computational procedure. Monte Carlo simulation results on the finite sample behavior of prediction-based model selection criteria P 2 are provided. We also evaluated two versions of the R 2 criterion. Finally, applications to real data are presented. The new criterion proved to be crucial to choose models taking into account the robustness of the maximum likelihood estimation procedure in the presence of influential cases.

A New Two-Parameter Estimator for Beta Regression Model: Method, Simulation, and Application

“…, such that ψ(•) denoting the digamma function, and g ′ (•) is the first derivative of g(•). The iterative reweighted least-squares (IWLS) algorithm or Fisher's scoring algorithm was used for estimating β [34,35]. The form of this algorithm can be written as:…”

“…ββ is the information matrix for β , as shown in References Espinheira et al [35] for more details. The initial value of β can be obtained by the least-squares estimation, while the initial value for each precision parameter is:…”

Developing a Liu‐type estimator in beta regression model

“…To get the estimated vector of , one can use the iterative reweighted least square (IWLS) method or the Fisher scoring method 41,42 . After a few iterations are performed, then convergence happens as the gap between consecutive estimates becomes less than a significant constant.…”

Beta regression model nonlinear in the parameters with additive measurement errors in variables

“…We have tried two different link functions: the logit and the complementary log-log. According to [17], when the mean of the response variable is near 1, the complementary log-log link function for the mean softens the impact of influent points in maximum likelihood estimation. In this sense, we consider here the logit and the complementary log-log link functions and check the performances of both modelings.…”

“…Many expressions in (17) differ only in the parameter of interest. More specifically, they differ as to the derivatives of η 1 and η 2 with respect to these parameters.…”