Elastic Net Regression and Empirical Mode Decomposition for Enhancing the Accuracy of the Model Selection

Abstract

Elastic net (ELNET) regression is a hybrid statistical technique used for regularizing and selecting necessary predictor variables that have a strong effect on the response variable and deal with multicollinearity problem when it exists between the predictor variables. The empirical mode decomposition (EMD) algorithm is used to decompose the nonstationary and nonlinear dataset into a finite set of orthogonal intrinsic mode function components and one residual component. This study mainly aims to apply the proposed ELNET-EMD method to determine the effect of the decomposition components of multivariate time-series predictors on the response variable and tackle the multicollinearity between the decomposition components to enhance the prediction accuracy for building a fitting model. A numerical experiment and a real data application are applied. Results show that the proposed ELNET-EMD method outperforms other existing methods by capable of identifying the decomposition components that have the most significance on the response variable despite the high correlation between the decomposition components and by improving the prediction accuracy.

A Comparative Approach on Bridge and Elastic Net Regressions

“…Reference [7], stated that Elastic net regression is a hybrid statistical technique used for regularizing and selecting necessary predictor variables that have a strong effect on the response variable and deal with multicollinearity problem when it exists between the predictor variables. Elastic Net can remove or select the predictor variables that have a high correlation in the final model and enhance the prediction accuracy.…”

“…Consequently, the behavior of the time series variables used in regression analysis, such as nonstationary and nonlinear, and multicollinearity problem may affect the prediction accuracy in model selection. [7]. Thus, better fitted models can help increase the accuracy of predictions and improve performance of estimates of regression coefficients.…”

A new shrinkage method for higher dimensions regression model to remedy of multicollinearity problem

“…The estimators of the Least Squares (LS) method have, of course, a clear and accurate interpretation, but in the case of the presence of some of the explanatory variables (p) that are not related to the response variable and are correlated with each other, then failure to exclude them leads to additional complications. Also, the estimators of the parameters resulting from the (LS) method are unlikely to be equal to Zero, which leads to the emergence of all variables in the model, and therefore the methodology of Regularization for explanatory variables has been studied for the purpose of excluding correlated variables according to accurate Statistical methodologies [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. In the same context, shrinkage is one of regularization methods that are taken for fitting a regression model using all (p) predictors, but under some constraint on the size of their estimated coefficients.…”

“…In the same context, shrinkage is one of regularization methods that are taken for fitting a regression model using all (p) predictors, but under some constraint on the size of their estimated coefficients. The importance of shrinkage lies in getting rid of the multicollinearity problem by reducing the variance of estimators in the model [1][2][3][4][5][6][7][8][9][10][11][12][13][14].…”

“…The methods described above are considered the most common and widely used because they lead to approximation of the values of coefficients of the variables in the model to zero or very close to Zero. [3,17,18].…”