Regression shrinkage and selection variables via an adaptive elastic net model

Abstract

In this paper, a new method of selection variables is presented to select some essential variables from large datasets. The new model is a modified version of the Elastic Net model. The modified Elastic Net variable selection model has been summarized in an algorithm. It is applied for Leukemia dataset that has 3051 variables (genes) and 72 samples. In reality, working with this kind of dataset is not accessible due to its large size. The modified model is compared to some standard variable selection methods. Perfect classification is achieved by applying the modified Elastic Net model because it has the best performance. All the calculations that have been done for this paper are in R program by using some existing packages.

Enhancing Bitcoin Price Volatility Estimator Predictions: A Four-Step Methodological Approach Utilizing Elastic Net Regression

“…This is particularly useful for models that benefit from variable reduction/selection. Similar to Ridge, the strength of the regularization is controlled by a hyperparameter, λ. Lasso regression satisfies the next optimization problem [34].…”

A new shrinkage method for higher dimensions regression model to remedy of multicollinearity problem

“…The General Linear Model (GLM) is one of the most widely used models in Various fields of statistical analysis, and the Ordinary Least Squares (OLS) method is one of the most common methods for estimating the parameters of the general linear model, as Follows: [7,12,18]…”

“…Therefore, based on this constraint, Lasso regression works to shrink the regression parameters and set them equal to Zero, as well as variables greater than Zero are determined after reduction and adopted as part of the model, which contributes to minimizing the prediction error and thus preserving the good features of both stepwise selection methodology and ridge regression method (RR). This method is of great importance to deal with the problem of multicollinearity between explanatory variables [7,8,11].…”

“…The estimators of the Least Squares (LS) method have, of course, a clear and accurate interpretation, but in the case of the presence of some of the explanatory variables (p) that are not related to the response variable and are correlated with each other, then failure to exclude them leads to additional complications. Also, the estimators of the parameters resulting from the (LS) method are unlikely to be equal to Zero, which leads to the emergence of all variables in the model, and therefore the methodology of Regularization for explanatory variables has been studied for the purpose of excluding correlated variables according to accurate Statistical methodologies [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. In the same context, shrinkage is one of regularization methods that are taken for fitting a regression model using all (p) predictors, but under some constraint on the size of their estimated coefficients.…”

“…In the same context, shrinkage is one of regularization methods that are taken for fitting a regression model using all (p) predictors, but under some constraint on the size of their estimated coefficients. The importance of shrinkage lies in getting rid of the multicollinearity problem by reducing the variance of estimators in the model [1][2][3][4][5][6][7][8][9][10][11][12][13][14].…”