Fast Algorithms and Theory for High-Dimensional Bayesian Varying Coefficient Models

Abstract

Nonparametric varying coefficient (NVC) models are widely used for modeling time-varying effects on responses that are measured repeatedly. In this paper, we introduce the nonparametric varying coefficient spike-and-slab lasso (NVC-SSL) for Bayesian estimation and variable selection in NVC models. The NVC-SSL simultaneously estimates the functionals of the significant time-varying covariates while thresholding out insignificant ones. Our model can be implemented using a highly efficient expectation-maximization (EM) algorithm, thus avoiding the computational burden of Markov chain Monte Carlo (MCMC) in high dimensions. In contrast to frequentist NVC models, hardly anything is known about the large-sample properties for Bayesian NVC models. In this paper, we take a step towards addressing this longstanding gap between methodology and theory by deriving posterior contraction rates under the NVC-SSL model when the number of covariates grows at nearly exponential rate with sample size. Finally, we illustrate our methodology through simulation studies and data analysis.

High-Dimensional Varying Coefficient Models with Functional Random Effects

“…From the plot for ABF1, the estimated function seems to follow the general shape of the estimate from (Bai et al, 2019), with both estimates entirely contained within the confidence bands. For MAC1, we observe that the estimated curves differ for the two estimation procedures, but the confidence band contains the majority of the estimated curve by NVC-SSL, with the discrepancies at the two endpoints.…”

“…Moreover, for n sufficiently large, Theorem 8 implies that P ∀t ∈ (0, 1) : l(t) ≤ β 1 (t) ≤ u(t) ≥ 1 − τ. . "Estimate" and "CB" are the estimate and confidence band from Section 5 respectively while "NVC-SSL" is the methodology from (Bai et al, 2019).…”

“…In this section, we apply our methodology to analyze transcription factors affecting the cell cycle of yeast. Versions of this data was previously analyzed in the high-dimensional varying coefficients framework by (Wei et al, 2011) and (Bai et al, 2019), to which we refer the interested reader for a more detailed description of the data. For our analysis, we use the data from (Bai et al, 2019), which consists of n = 47 genes and p = 96 transcription factors.…”

“…(Lee & Mammen, 2016)). The work most similar to ours is (Bai, Boland & Chen, 2019), who assume m i observations for individual, i = 1, . .…”

“…Figure5: Marginal confidence bands for ABF1 and MAC1. "Estimate" and "CB" are the estimate and confidence band from Section 5 respectively while "NVC-SSL" is the methodology from(Bai et al, 2019).…”