Bayesian wavelet estimation of partially linear models

Abstract

A Bayesian wavelet approach is presented for estimating a partially linear model (PLM). A PLM consists of a linear part and a nonparametric component. The nonparametric component is represented with a wavelet series where the wavelet coefficients have assumed prior distributions. The prior for each coefficient consists of a mixture of a normal distribution and a point mass at 0. The linear parameters are assumed to have a normal prior. The hyperparameters are estimated by the marginal maximum likelihood estimator using the direct maximization. The model selection and model averaging methods give different estimates of the model parameters. MCMC computation is used for the estimation of the linear coefficients by model averaging method. Simulated examples illustrate the performance of the proposed estimators.

Fully Bayesian Estimation and Variable Selection in Partially Linear Wavelet Models

“…The first one is the wavelet Backfitting algorithm (BF ) proposed by Chang and Qu (2004), the second one is the LEGEND algorithm proposed by Gannaz (2007) and the last one is the double penalized PLM wavelet estimator (DPPLM ) by Ding et al (2011). A Bayesian wavelet-based algorithm for the same problem was proposed by Qu (2006). However, we found that the implementation of that algorithm is not robust to different simulated examples and initial values of the empirical Bayes procedure, therefore, we omitted it from our discussion.…”

“…Another efficient way to elicit the hyperparameters of the model is through the empirical Bayes method performing maximization of the marginal likelihood. This approach was followed by Qu (2006) in the context of estimating partially linear wavelet models.…”

“…These all build on existing nonparametric regression techniques, such as the kernel method, the local linear method (local polynomial or trigonometric polynomial techniques), or splines. In the most recent papers, wavelets are used (Chang and Qu, 2004;Fadili and Bullmore, 2005;Qu, 2006;Gannaz, 2007;Ding et al, 2011), which allows the nonparametric component to be parsimoniously represented by a limited number of coefficients. The wavelet representation can include a wide variety of nonparametric parts, including non-smooth signals, and reduces the bias in estimating the parametric component.…”

“…We use the Bayesian approach to formulate a hierarchical model in the wavelet domain and estimate its parameters. Only a few papers used wavelets in the partially linear model context, and besides Qu (2006), all used a penalized least squares estimation procedure. Therefore, using a fully Bayesian approach can be of interest.…”

A note on Bayes factor consistency in partial linear models

“…Bayesian approaches to the partial linear models have been studied in the literature by developing different methods for estimating the nonparametric component f (·) (e.g. the Fourier series Lenk, 1999 andChoi et al, 2009), splines (Fahrmeir et al, 2013;Chib and Greenberg, 2010;Kyung, 2011), Gaussian processes (Choi and Woo, 2015), smoothing priors (Koop and Poirier, 2004), and wavelets (Qu, 2006;Ko et al, 2009). Here, we study specific partial linear models using two different families of Gaussian processes, which are very common nonparametric priors for Bayesian regression functions.…”