Automatic Search Interval for Smoothing Parameter in Penalized Splines

Abstract

The selection of smoothing parameter is central to estimation of penalized splines. The best parameter value is often the one that optimizes a smoothness selection criterion, like the minimizer of generalized cross-validation error (GCV) and the maximizer of restricted likelihood (REML). To avoid ending up with an undesired local extremum rather than the global extremum, grid search should be used for optimization. Unfortunately, the method requires a pre-specified search interval that contains the unknown global extremum and there has not been any theory on how it could be provided. As a result, practitioners have to find it by trial and error. To overcome such difficulty, we develop novel algorithms to automatically find this interval. Our automatic search interval has four advantages. (i) It specifies a smoothing parameter range where the penalized least squares problem is numerically solvable. (ii) It is criterion-independent, so that different criteria like GCV and REML can be explored on the same parameter range. (iii) It is sufficiently wide to contain the global extremum of any criterion, so that for example, the global minimum of GCV and the global maximum of REML can both be identified. (iv) It is computationally cheap compared with grid search so that it carries no extra costs in practice. Our method is ready to use through R package gps (>= version 1.1). It may be embedded in other advanced statistical modeling methods that rely on penalized splines.