The lasso for high-dimensional regression with a possible change-point

Lee, Sokbae; Seo, Myung Hwan; Shin, Youngki

doi:10.1920/wp.cem.2014.2614

2014

DOI: 10.1920/wp.cem.2014.2614

|View full text |Cite

The lasso for high-dimensional regression with a possible change-point

Sokbae Lee¹,

Myung Hwan Seo²,

Youngki Shin³

Abstract: Summary. We consider a high dimensional regression model with a possible change point due to a covariate threshold and develop the lasso estimator of regression coefficients as well as the threshold parameter. Our lasso estimator not only selects covariates but also selects a model between linear and threshold regression models. Under a sparsity assumption, we derive non-asymptotic oracle inequalities for both the prediction risk and the l 1 -estimation loss for regression coefficients. Since the lasso estimat… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2018

2021

Publication Types

Select...

Other3

Relationship

Self Cite1

Independent2

Authors

Journals

Cited by 3 publications

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Multi-threshold Change Plane Model: Estimation Theory and Applications in Subgroup Identification

Li¹,

Li²,

Jin³

2018

Preprint

View full text Add to dashboard Cite

We propose a multi-threshold change plane regression model which naturally partitions the observed subjects into subgroups with different covariate effects. The underlying grouping variable is a linear function of covariates and thus multiple thresholds form parallel change planes in the covariate space. We contribute a novel 2-stage approach to estimate the number of subgroups, the location of thresholds and all other regression parameters. In the first stage we adopt a group selection principle to consistently identify the number of subgroups, while in the second stage change point locations and model parameter estimates are refined by a penalized induced smoothing technique. Our procedure allows sparse solutions for relatively moderate-or high-dimensional covariates. We further establish the asymptotic properties of our proposed estimators under appropriate technical conditions. We evaluate the performance of the proposed methods by simulation studies and provide illustration using two medical data. Our proposal for subgroup identification may lead to an immediate application in personalized medicine.

show abstract

Multi-threshold Change Plane Model: Estimation Theory and Applications in Subgroup Identification

Li¹,

Li²,

Jin³

2018

Preprint

View full text Add to dashboard Cite

show abstract

Statistically and Computationally Efficient Change Point Localization in Regression Settings

Wang

Lin

Willett

2019

Preprint

View full text Add to dashboard Cite

Detecting when the underlying distribution changes from the observed time series is a fundamental problem arising in a broad spectrum of applications. Change point localization is particularly challenging when we only observe low-dimensional projections of high-dimensional random variables. Specifically, we assume we observe {x t , y t } n t=1 where {x t } n t=1 are p-dimensional covariates, {y t } n t=1 are the univariate responses satisfying E(y t ) = x ⊤ t β * t for all 1 ≤ t ≤ n and that {β * t } n t=1 are the unobserved regression parameters that change over time in a piecewise constant manner. We first propose a novel algorithm called Binary Segmentation through Estimated CUSUM statistics (BSE), which computes the change points through direct estimates of the CUSUM statistics of {β * t } n t=1. We show that BSE can consistently estimate the unknown location of the change points, achieving error bounds of order O(log(p)/n). To the best of our knowledge, this is a significant improvement, as the state-of-the-art methods are only shown to achieve error bounds of order O(log(p)/ √ n) in the multiple change point setting. However, BSE can be computationally costly when the number change points is large. To overcome this limitation, we introduce another new algorithm called Binary Segmentation through Lasso Estimators (BSLE). We show that BSLE can consistently localize change points with a slightly worse localization error rate compared to BSE, but BSLE is much more computationally efficient. Finally, we leverage the insights gained from BSE and BSLE to develop a novel "local screening" algorithm that can input a coarse estimate of change point locations together with the observed data and efficiently refine that estimate, allowing us to improve the practical performance of past estimators based on group lasso. All of our newly proposed algorithms have good performance in our simulated experiments, especially when the size of changes in the regression parameters {β * t } n t=1 is small.

show abstract

Regression Discontinuity Design with Potentially Many Covariates

Arai¹,

Otsu²,

Seo³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper studies the case of possibly high-dimensional covariates in the regression discontinuity design (RDD) analysis. In particular, we propose estimation and inference methods for the RDD models with covariate selection which perform stably regardless of the number of covariates. The proposed methods combine the local approach using kernel weights with ℓ 1 -penalization to handle high-dimensional covariates, and the combination is new in the literature. We provide theoretical and numerical results which illustrate the usefulness of the proposed methods. Theoretically, we present risk and coverage properties for our point estimation and inference methods, respectively. Numerically, our simulation experiments and empirical example show the robust behaviors of the proposed methods to the number of covariates in terms of bias and variance for point estimation and coverage probability and interval length for inference.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

The lasso for high-dimensional regression with a possible change-point

Cited by 3 publications

References 18 publications

Multi-threshold Change Plane Model: Estimation Theory and Applications in Subgroup Identification

Multi-threshold Change Plane Model: Estimation Theory and Applications in Subgroup Identification

Statistically and Computationally Efficient Change Point Localization in Regression Settings

Regression Discontinuity Design with Potentially Many Covariates

Contact Info

Product

Resources

About