We consider estimation and inference in a single index regression model with an unknown but smooth link function. In contrast to the standard approach of using kernel methods, we use smoothing splines to estimate the smooth link function. We develop a method to compute the penalized least squares estimators (PLSEs) of the parametric and the nonparametric components given independent and identically distributed (i.i.d.) data. We prove the consistency and find the rates of convergence of the estimators. We establish n −1/2 -rate of convergence and the asymptotic efficiency of the parametric component under mild assumptions. A finite sample simulation corroborates our asymptotic theory and illustrates the superiority of our procedure over existing procedures. We also analyze a car mileage data set and a ozone concentration data set. The identifiability and existence of the PLSEs are also investigated.
We discuss a model-robust theory for general types of regression in the simplest case of iid observations. The theory replaces the parameters of parametric models with statistical functionals, to be called "regression functionals" and defined on large non-parametric classes of joint x-y distributions without assuming a working model. Examples of regression functionals are the slopes of OLS linear equations at largely arbitrary x-y distributions (see Part I). More generally, regression functionals can be defined by minimizing objective functions or solving estimating equations at joint x-y distributions. The role of parametric models is reduced to heuristics for generating objective functions and estimating equations without assuming them as correct. In this framework it is possible to achieve the following: (1) explicate the meaning of mis/well-specification for regression functionals, (2) decompose sampling variability into two components, one due to the conditional response distributions and another due to the regressor distribution interacting (conspiring) with misspecification, (3) exhibit plug-in (and hence sandwich) estimators of standard error as limiting cases of x-y bootstrap estimators.
Concentration inequalities form an essential toolkit in the study of high-dimensional statistical methods. Most of the relevant statistics literature in this regard is, however, based on the assumptions of sub-Gaussian or sub-exponential random variables/vectors. In this paper, we first bring together, through a unified exposition, various probabilistic inequalities for sums of independent random variables under much more general exponential type (namely sub-Weibull) tail assumptions. These results extract a part sub-Gaussian tail behavior of the sum in finite samples, matching the asymptotics governed by the central limit theorem, and are compactly represented in terms of a new Orlicz quasi-norm—the Generalized Bernstein–Orlicz norm—that typifies such kind of tail behaviors. We illustrate the usefulness of these inequalities through the analysis of four fundamental problems in high-dimensional statistics. In the first two problems, we study the rate of convergence of the sample covariance matrix in terms of the maximum elementwise norm and the maximum $k$-sub-matrix operator norm that are key quantities of interest in bootstrap procedures and high-dimensional structured covariance matrix estimation, as well as in high-dimensional and post-selection inference. The third example concerns the restricted eigenvalue condition, required in high-dimensional linear regression, which we verify for all sub-Weibull random vectors through a unified analysis, and also prove a more general result related to restricted strong convexity in the process. In the final example, we consider the Lasso estimator for linear regression and establish its rate of convergence to be generally $\sqrt{k\log p/n}$, for $k$-sparse signals, under much weaker than usual tail assumptions (on the errors as well as the covariates), while also allowing for misspecified models and both fixed and random design. To our knowledge, these are the first such results for Lasso obtained in this generality. The common feature in all our results over all the examples is that the convergence rates under most exponential tails match the usual (optimal) ones obtained under sub-Gaussian assumptions. Finally, we also establish some complementary results on analogous tail bounds for the suprema of empirical processes indexed by sub-Weibull variables. All our results are finite samples.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.