Bias-corrected inference for multivariate nonparametric regression: Model selection and oracle property

Abstract

The local polynomial estimator is particularly affected by the curse of dimensionality, which reduces the potential of this tool for large-dimensional applications. We propose an estimation procedure based on the local linear estimator and a sparseness condition that focuses on nonlinearities in the model. Our procedure, called BID (bias inflation--deflation), is automatic and easily applicable to models with many covariates without requiring any additivity assumption. It is an extension of the RODEO method, and introduces important new contributions: consistent estimation of the multivariate optimal bandwidth (the tuning parameter of the estimator); consistent estimation of the multivariate bias-corrected regression function and confidence bands; and automatic identification and separation of nonlinear and linear effects. Some theoretical properties of the method are discussed. In particular, we show the nonparametric oracle property. For linear models, BID automatically reaches the optimal rate $O_p(n^{-1/2})$, equivalent to the parametric case. A simulation study shows the performance of the procedure for finite samples

Variable selection in high‐dimensional regression: a nonparametric procedure for business failure prediction

“…In business failure modeling, the traditional and modern approaches to variable selection (i.e., stepwise regression, allsubset regression, Lasso, and Adaptive Lasso) have several drawbacks, as explained in Sections 1 and 2, as they are based on fundamental assumptions of linearity and additivity that are not verified in business failure prediction modeling. For this reason, as anticipated in Section 2, we propose to select the predictors by a nonparametric procedure, the Rodeo of [34], recently extended in [35], that is described in Section 3.2. In Section 3.1, we briefly introduce Lasso and Adaptive Lasso, two variable selection methods that we compare our proposed procedure with.…”

“…The suggested method, named Regularization of Derivative Expectation Operator (Rodeo), has been proposed by [34] to overcome the well-known problem of nonparametric multivariate regression, called curse of dimensionality. Recently, [35] extended Rodeo in two directions: they relaxed some of the theoretical assumptions underlying it, and they improved the final nonparametric regression by deriving (and estimating) the optimal multivariate bandwidth matrix using the Bias-Inflation-Deflation method. Therefore, the Rodeo (combined with Bias-Inflation-Deflation method) procedure has the important advantage of being easily implemented, when it is used for nonparametric regression, because the multivariate function estimation is automatically optimized by a data-driven procedure that does not depend on pilot tuning parameters, as shown in [35].…”

“…Recently, [35] extended Rodeo in two directions: they relaxed some of the theoretical assumptions underlying it, and they improved the final nonparametric regression by deriving (and estimating) the optimal multivariate bandwidth matrix using the Bias-Inflation-Deflation method. Therefore, the Rodeo (combined with Bias-Inflation-Deflation method) procedure has the important advantage of being easily implemented, when it is used for nonparametric regression, because the multivariate function estimation is automatically optimized by a data-driven procedure that does not depend on pilot tuning parameters, as shown in [35]. Moreover, when Rodeo is used for variable selection (instead of smoothing regression), as is the case here, the robustness of the estimation results (with respect to the choice of tuning parameters) is even stronger, because we do not need any optimized bandwidth matrix in the variable selection stage.…”

“…Because the Rodeo procedure may be blind to linearities and can miss relevant linear covariates, it cannot be used for variable selection. An extension of this procedure has been recently proposed by [35] to fix the problem and to assure the nonparametric oracle property with respect to both variable selection and multivariate function estimation. In this section, we briefly describe the (extended) Rodeo procedure for variable selection.…”

“…In this section, we briefly describe the (extended) Rodeo procedure for variable selection. Further details may be found in [34] and [35]. Appl.…”