The distribution of the Lasso: Uniform control over sparse balls and adaptive parameter tuning

Leo, Marco; Montanari, Andrea

doi:10.1214/20-aos2038

Cited by 28 publications

(8 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We present Propositions 3.14 and 3.15 as example results to show the verification of our assumptions follow rather easily from the existing asymptotic profile results in the literature. In the proportional asymptotic regime, the risk profiles have been characterized for various other prediction procedures including, high dimensional robust M -estimator (Karoui, 2013(Karoui, , 2018Donoho and Montanari, 2016), the Lasso estimator (Miolane and Montanari, 2021;Celentano et al, 2020), and various classification procedures (Montanari et al, 2019;Liang and Sur, 2020;Sur et al, 2019). Our results can be suitably extended to verify (DETPA-0) for these other predictors.…”

Section: Verifying Deterministic Profile Assumption (Detpar-0)mentioning

confidence: 78%

Mitigating multiple descents: A model-agnostic framework for risk monotonization

Patil¹,

Kuchibhotla²,

Wei³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent empirical and theoretical analyses of several commonly used prediction procedures reveal a peculiar risk behavior in high dimensions, referred to as double/multiple descent, in which the asymptotic risk is a non-monotonic function of the limiting aspect ratio of the number of features or parameters to the sample size. To mitigate this undesirable behavior, we develop a general framework for risk monotonization based on cross-validation that takes as input a generic prediction procedure and returns a modified procedure whose out-of-sample prediction risk is, asymptotically, monotonic in the limiting aspect ratio. As part of our framework, we propose two data-driven methodologies, namely zero-and one-step, that are akin to bagging and boosting, respectively, and show that, under very mild assumptions, they provably achieve monotonic asymptotic risk behavior. Our results are applicable to a broad variety of prediction procedures and loss functions, and do not require a well-specified (parametric) model. We exemplify our framework with concrete analyses of the minimum 2, 1-norm least squares prediction procedures. As one of the ingredients in our analysis, we also derive novel additive and multiplicative forms of oracle risk inequalities for split cross-validation that are of independent interest.

show abstract

Section: Verifying Deterministic Profile Assumption (Detpar-0)mentioning

confidence: 78%

Mitigating multiple descents: A model-agnostic framework for risk monotonization

Patil¹,

Kuchibhotla²,

Wei³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In Section 4, we have presented a general recipe that uses AMP systematically to obtain exact expressions for the asymptotic error of penalised and unpenalised M-estimators in GLMs with Gaussian design matrices. An alternative approach to deriving such guarantees is via Gaussian comparison inequalities and the convex Gaussian min-max theorem (CGMT); see for instance Thrampoulidis et al (2015Thrampoulidis et al ( , 2018, Miolane and Montanari (2018) and Liang and Sur (2020) for applications of these techniques to regularised M-estimators, the Lasso and boosting respectively.…”

Section: Discussionmentioning

confidence: 99%

A unifying tutorial on Approximate Message Passing

Feng¹,

Venkataramanan²,

Rush³

et al. 2021

Preprint

View full text Add to dashboard Cite

Over the last decade or so, Approximate Message Passing (AMP) algorithms have become extremely popular in various structured high-dimensional statistical problems. The fact that the origins of these techniques can be traced back to notions of belief propagation in the statistical physics literature lends a certain mystique to the area for many statisticians. Our goal in this work is to present the main ideas of AMP from a statistical perspective, to illustrate the power and flexibility of the AMP framework. Along the way, we strengthen and unify many of the results in the existing literature.

show abstract

“…By Lemma F.14 in [28], function x → a(x) is γ n -strongly convex on the ball B for some γ > 0. Particularly the hessian of a satisfies:…”

Section: A Proof Of Lemmamentioning

confidence: 95%

“…Finally, plugging the above expressions into the expressions of P ⋆ b , P ⋆ d , P ⋆ e and SINR ⋆ lb yields the convergences in ( 25)- (28).…”

Section: A Proof Of Theoremmentioning

confidence: 99%

Sharp Analysis of RLS-based Digital Precoder with Limited PAPR in Massive MIMO

Ma,

Kammoun,

Alrashdi

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper focuses on the performance analysis of a class of limited peak-to-average power ratios (PAPR) precoders for downlink multi-user massive multiple-input multiple-output (MIMO) systems. Contrary to conventional precoding approaches based on simple linear precoders maximum ratio transmission (MRT) and regularized zero forcing (RZF), the precoders in this paper are obtained by solving a convex optimization problem. To be specific, for the precoders we analyze in this paper, the power of each precoded symbol entry is restricted, which allows them to present a reduced PAPR at each antenna. By using the Convex Gaussian Min-max Theorem (CGMT), we analytically characterize the empirical distribution of the precoded vector and the joint empirical distribution between the distortion and the intended symbol vector. This allows us to study the performance of these precoders in terms of perantenna power, per-user distortion power, signal to interference and noise ratio, and bit error probability.We show that for this class of precoders, there is an optimal transmit power that maximizes the system performance.

show abstract

The distribution of the Lasso: Uniform control over sparse balls and adaptive parameter tuning

Cited by 28 publications

References 42 publications

Mitigating multiple descents: A model-agnostic framework for risk monotonization

Mitigating multiple descents: A model-agnostic framework for risk monotonization

A unifying tutorial on Approximate Message Passing

Sharp Analysis of RLS-based Digital Precoder with Limited PAPR in Massive MIMO

Contact Info

Product

Resources

About