The impact of collinearity on regression analysis: the asymmetric effect of negative and positive correlations

Abstract

The purpose of this paper is to ascertain how collinearity in general, and the sign of correlations in specific, affect parameter inference, variable omission bias, and their diagnostic indices in regression. It is found that collinearity can reduce parameter variance estimates and that positive and negative correlation structures have an asymmetric effect on variable omission bias. It is also shown that the effects of collinearity are moderated by the relationship between the dependent variable and the regressors, a consideration not incorporated into most commonly used collinearity diagnostics. The formulae derived enable researchers to assess the sensitivity of regression results to the underlying correlation structure in the data.

Element interactions and soil properties affecting the soil-to-plant transfer of six elements relevant to radioactive waste in boreal forest

“…Multi-collinearity is a common problem related to regression analyses carried out with a large number of variables; correlations between variables can affect the results of the analysis (Mela and Kopalle 2002;Lin 2008). PCA and sensitivity analysis (by removing variables from the equations) were used in the present study as tools to interpret the results.…”

An assessment of the performance of a 1.5 μm Doppler lidar for operational vertical wind profiling based on a 1-year trial

“…This is achieved when the degree of collinearity among the Doppler velocity measurements used for the retrieval is relatively weak, since a robust linear independence of the sampling directions is an essential prerequisite for the reconstruction of the wind vector. Multicollinearity describes a high linear relationship among one or more independent variables (Belsley et al, 1980) and it is also a well known issue in regression analysis that multicollinearity may result in parameter estimates with incorrect signs and implausible magnitudes (Mela and Kopalle, 2002) or may affect the regressions' robustness, i.e. small changes in the data may result in large changes in the parameter estimates (Boccippio, 1995).…”

The More the Merrier? The Effects of Community Feedback on Idea Quality in Innovation Contests

“…All variables in the regression model were standardized to account for differing scales and avoid inflation in multicolinearity. Correlation analysis (see Table 2) indicate some high bivariate correlations (>0.7), suggesting potential multicollinearity [32]. We examined the variance inflation factor (VIF) to determine if any of the variables should be dropped from our analysis due to multicollinearity.…”