Bootstrapping Lasso Estimators

Chatterjee, Arindam; Lahiri, Soumendra N.

doi:10.1198/jasa.2011.tm10159

Cited by 245 publications

(235 citation statements)

References 24 publications

Supporting

Mentioning

233

Contrasting

Order By: Relevance

“…All of these procedures yield qualitatively similar conclusions. Note that we do not report results for a standard nonparametric bootstrap; the standard nonparametric bootstrap is known to be invalid for lasso regression (Chatterjee and Lahiri 2011).…”

Section: Inferencementioning

confidence: 99%

Measuring Group Differences in High-Dimensional Choices: Method and Application to Congressional Speech

Gentzkow¹,

Shapiro²,

Taddy

2016

View full text Add to dashboard Cite

for their comments and suggestions. We thank Frances Lee for sharing her data on congressional communications staff. We also thank numerous seminar audiences and our many dedicated research assistants for their contributions to this project. This work was completed in part with resources provided by the University of Chicago Research Computing Center and the Stanford Research Computing Center. The data providers and funding agencies bear no responsibility for use of the data or for interpretations or inferences based upon such uses. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.At least one co-author has disclosed a financial relationship of potential relevance for this research. Further information is available online at http://www.nber.org/papers/w22423.ack NBER working papers are circulated for discussion and comment purposes. They have not been peer-reviewed or been subject to the review by the NBER Board of Directors that accompanies official NBER publications.© 2016 by Matthew Gentzkow, Jesse M. Shapiro, and Matt Taddy. All rights reserved. Short sections of text, not to exceed two paragraphs, may be quoted without explicit permission provided that full credit, including © notice, is given to the source. ABSTRACTWe study the problem of measuring group differences in choices when the dimensionality of the choice set is large. We show that standard approaches suffer from a severe finite-sample bias, and we propose an estimator that applies recent advances in machine learning to address this bias. We apply this method to measure trends in the partisanship of congressional speech from 1873 to 2016, defining partisanship to be the ease with which an observer could infer a congressperson's party from a single utterance. Our estimates imply that partisanship is far greater in recent years than in the past, and that it increased sharply in the early 1990s after remaining low and relatively constant over the preceding century.

show abstract

Section: Inferencementioning

confidence: 99%

Measuring Group Differences in High-Dimensional Choices: Method and Application to Congressional Speech

Gentzkow¹,

Shapiro²,

Taddy

2016

View full text Add to dashboard Cite

show abstract

“…Methods such as adaptive Lasso and boostrapping Lasso have been proposed for these cases [1], [3], [13], [15]. Building on work done for the linear model in Equation 1 [4], [9], in this paper, we show that Lasso is consistent when only part of the underlying function is linear as p and n grow large.…”

Section: Introductionmentioning

confidence: 93%

Resampling-Based Variable Selection with Lasso for p >> n and Partially Linear Models

Mares

Guo

2015

2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA)

View full text Add to dashboard Cite

The linear model of the regression function is a widely used and perhaps, in most cases, highly unrealistic simplifying assumption, when proposing consistent variable selection methods for large and highly-dimensional datasets. In this paper, we study what happens from theoretical point of view, when a variable selection method assumes a linear regression function and the underlying ground-truth model is composed of a linear and a non-linear term, that is at most partially linear. We demonstrate consistency of the Lasso method when the model is partially linear. However, we note that the algorithm tends to increase even more the number of selected false positives on partially linear models when given few training samples. That is usually because the values of small groups of samples happen to explain variation coming from the non-linear part of the response function and the noise, using a linear combination of wrong predictors. We demonstrate theoretically that false positives are likely to be selected by the Lasso method due to a small proportion of samples, which happen to explain some variation in the response variable. We show that this property implies that if we run the Lasso on several slightly smaller size data replications, sampled without replacement, and intersect the results, we are likely to reduce the number of false positives without losing already selected true positives. We propose a novel consistent variable selection algorithm based on this property and we show it can outperform other variable selection methods on synthetic datasets of linear and partially linear models and datasets from the UCI machine learning repository.

show abstract

“…The Adaptive Lasso ensures that the bootstrap (Section 3. 3) is consistent (Chatterjee and Lahiri, 2011). We take the weights of the Adaptive Lassô…”

Section: Penalized Maximum Likelihood Estimationmentioning

confidence: 99%

“…Recently, a small but growing literature on inference in penalized regression models for cross-sectional data has arisen, such as Wasserman and Roeder (2009), Meinshausen et al (2009) and Chatterjee and Lahiri (2011). We extend the residual bootstrap procedure of Chatterjee and Lahiri (2011) to high-dimensional time series data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Predictive Power of the Business and Bank Sentiment of Firms: A High-Dimensional Granger Causality Approach

2015

View full text Add to dashboard Cite

Abstract. We study the predictive power of industry-specific economic sentiment indicators for future macro-economic developments. In addition to the sentiment of firms towards their own business situation, we study their sentiment with respect to the banking sector -their main credit providers. The use of industry-specific sentiment indicators results in a high-dimensional forecasting problem. To identify the most predictive industries, we present a bootstrap Granger Causality test based on the Adaptive Lasso. This test is more powerful than the standard Wald test in such high-dimensional settings. Forecast accuracy is improved by using only the most predictive industries rather than all industries.

show abstract

Bootstrapping Lasso Estimators

Cited by 245 publications

References 24 publications

Measuring Group Differences in High-Dimensional Choices: Method and Application to Congressional Speech

Measuring Group Differences in High-Dimensional Choices: Method and Application to Congressional Speech

Resampling-Based Variable Selection with Lasso for p >> n and Partially Linear Models

The Predictive Power of the Business and Bank Sentiment of Firms: A High-Dimensional Granger Causality Approach

Contact Info

Product

Resources

About