Handling Critical Multicollinearity Using Parametric Approach

Abstract

In regression analysis, it is relatively necessary to have a correlation between the response and explanatory variables, but having correlations amongst explanatory variables is something undesired. This paper focuses on five methodologies for handling critical multicollinearity, they include: Partial Least Square Regression (PLSR), Ridge Regression (RR), Ordinary Least Square Regression (OLS), Least Absolute Shrinkage and Selector Operator (LASSO) Regression, and the Principal Component Analysis (PCA). Monte Carlo Simulations comparing the methods was carried out with the sample size greater than or equal to the levels (n>p) considered in most cases, the Average Mean Square Error (AMSE) and Akaike Information Criterion (AIC) values were computed. The result shows that PCR is the most superior and more efficient in handling critical multicollinearity problems, having the lowest AMSE and AIC values for all the sample sizes and different levels considered.

Shiny App to Predict Agricultural Tire Dimensions

“…One possibility to correct this problem is using Principal Component Regression (PCR), which is a linear regression using principal components. Maxwell et al [ 7 ] wrote an article to tackle with multicollinearity effects and here 5 methodologies were tested: Partial Least Square Regression (PLSR), Ridge Regression (RR), Ordinary Least Square Regression (OLS), Least Absolute Shrinkage and Selector Operator (LASSO) Regression, and the Principal Component Regression (PCR). To compare the 5 methodologies, they used a different number of observations and a number of predictor variables.…”

“…Root Mean Square Error (RMSE) and AIC were used to compare the performance of each model. With this analysis the authors concluded that PCR has the lowest AMSE and AIC, which means that according to them, PCR is the most efficient in handling critical multicollinearity effects [ 7 ]. Lafi and Kaneene used Principal Component Analysis (PCA) to detect and correct multicollinearity effects in a veterinary epidemiological study.…”