Case influence diagnostics in the lasso regression

Kim, Choongrak; Lee, Jungsu; Yang, Hojin; Bae, Whasoo

doi:10.1016/j.jkss.2014.09.003

Cited by 11 publications

(9 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By examining the various plots in Figures , we can gain better understanding of the impact of each influential observation on the estimated coefficients, as well as on the LASSO plot. The identification of the 69th, 95th, 96th and 97th observations as influential is similar to the results in Kim et al…”

Section: Influence Plots For Evaluating the Effect Of Influential Obssupporting

confidence: 89%

“…Detecting influential observations should be an important part of model selection and fitting, because influential observations can greatly impact results and reduce the effectiveness of the LASSO method. Recently, Zhao et al suggested high‐dimensional influence measures, and Kim et al proposed case influence diagnostics for LASSO regression. A LASSO plot can be a useful graphical tool for finding variable selection patterns in the presence of multicollinearity.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Influence Plots for LASSO

Jang

Anderson‐Cook

2016

Quality & Reliability Eng

View full text Add to dashboard Cite

With many predictors in regression, fitting the full model can induce multicollinearity problems. Least Absolute Shrinkage and Selection Operation (LASSO) is useful when the effects of many explanatory variables are sparse in a highdimensional dataset. Influential points can have a disproportionate impact on the estimated values of model parameters. This paper describes a new influence plot that can be used to increase understanding of the contributions of individual observations and the robustness of results. This can serve as a complement to other regression diagnostics techniques in the LASSO regression setting. Using this influence plot, we can find influential points and their impact on shrinkage of model parameters and model selection. We illustrate the methods with two examples.

show abstract

Section: Influence Plots For Evaluating the Effect Of Influential Obssupporting

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Influence Plots for LASSO

Jang

Anderson‐Cook

2016

Quality & Reliability Eng

View full text Add to dashboard Cite

show abstract

“…Kim et al (2015) recently derived an approximate version of Cook's distance in the LASSO regression; however, it is based on the "large n, small p" assumption. Cook's distance in the LASSO model suggested by Kim et al (2015) cannot be directly used in high-dimensional data because the covariance estimator of the LASSO estimator, which is necessary in defining a version of Cook's distance, is not easily derived in the high-dimensional data. Further, under high dimensional setup, we are more interested in influential observations on the variable selection rather than the influence on estimators.…”

Section: Introductionmentioning

confidence: 99%

Effect of outliers on the variable selection by the regularized regression

Jeong

Kim

2018

CSAM

Self Cite

View full text Add to dashboard Cite

Many studies exist on the influence of one or few observations on estimators in a variety of statistical models under the "large n, small p" setup; however, diagnostic issues in the regression models have been rarely studied in a high dimensional setup. In the high dimensional data, the influence of observations is more serious because the sample size n is significantly less than the number variables p. Here, we investigate the influence of observations on the least absolute shrinkage and selection operator (LASSO) estimates, suggested by Tibshirani (Journal of the Royal Statistical Society, Series B, 73, 273-282, 1996), and the influence of observations on selected variables by the LASSO in the high dimensional setup. We also derived an analytic expression for the influence of the k observation on LASSO estimates in simple linear regression. Numerical studies based on artificial data and real data are done for illustration. Numerical results showed that the influence of observations on the LASSO estimates and the selected variables by the LASSO in the high dimensional setup is more severe than that in the usual "large n, small p" setup.

show abstract

“…Diagnostic results in Box-Cox transformation model (Box and Cox, 1964) have also been done by Cook and Wang (1983), Hinkley and Wang (1988), Kim et al (1996), and Tsai and Wu (1990). Walker and Birch (1988) derived a version of Cook's distance in the ridge regression (Hoerl and Kennard, 1970), and Kim et al (2015) proposed a type of Cook's distance in the lasso regression model.…”

Section: Introductionmentioning

confidence: 99%

Case influence diagnostics for the significance of the linear regression model

Bae

Noh

Kim

2017

CSAM

Self Cite

View full text Add to dashboard Cite

In this paper we propose influence measures for two basic goodness-of-fit statistics, the coefficient of determination R 2 and test statistic F in the linear regression model using the deletion method. Some useful lemmas are provided. We also express the influence measures in terms of basic building blocks such as residual, leverage, and deviation that showed them as increasing function of residuals and a decreasing function of deviation. Further, the proposed measure reduces computational burden from O(n) to O(1). As illustrative examples, we applied the proposed measures to the stackloss data sets. We verified that deletion of one or few influential observations may result in big change in R 2 and F-statistic.

show abstract

Case influence diagnostics in the lasso regression

Cited by 11 publications

References 27 publications

Influence Plots for LASSO

Influence Plots for LASSO

Effect of outliers on the variable selection by the regularized regression

Case influence diagnostics for the significance of the linear regression model

Contact Info

Product

Resources

About