A generative nonparametric Bayesian model for whole genomes

Abstract

Generative probabilistic modeling of biological sequences has widespread existing and potential use across biology and biomedicine, particularly given advances in high-throughput sequencing, synthesis and editing. However, we still lack methods with nucleotide resolution that are tractable at the scale of whole genomes and that can achieve high predictive accuracy either in theory or practice. In this article we propose a new generative sequence model, the Bayesian embedded autoregressive (BEAR) model, which uses a parametric autoregressive model to specify a conjugate prior over a nonparametric Bayesian Markov model. We explore, theoretically and empirically, applications of BEAR models to a variety of statistical problems including density estimation, robust parameter estimation, goodness-of-fit tests, and two-sample tests. We prove rigorous asymptotic consistency results including nonparametric posterior concentration rates. We scale inference in BEAR models to datasets containing tens of billions of nucleotides. On genomic, transcriptomic, and metagenomic sequence data we show that BEAR models provide large increases in predictive performance as compared to parametric autoregressive models, among other results. BEAR models offer a flexible and scalable framework, with theoretical guarantees, for building and critiquing generative models at the whole genome scale.

Non-identifiability and the Blessings of Misspecification in Models of Molecular Fitness

“…We embedded the AR model (a convolutional neural network) into a BEAR model, and fit the BEAR model with empirical Bayes. We found evidence that the AR model was misspecified on every dataset, following the methodology of Amin, Weinstein and Marks [2]: the optimal h selected by empirical Bayes was on the order of 1 − 10 in each dataset. Now, in the limit as the hyperparameter h →0, the BEAR model collapses to its embedded AR model; so by scanning h from low to high values we can interpolate between the parametric and nonparametric regime.…”

“…To accomplish this, we compute a posterior over p 0 using a Bayesian nonparametric sequence model. In particular, we apply the Bayesian embedded autoregressive (BEAR) model, which can be scaled to terabytes of data and satisfies posterior consistency ([2], Thm. 35):…”

“…e. for some t > 0, , where | X | is the length of sequence X. Assume the conditions on the prior detailed in Amin, Weinstein and Marks [2]. If X 1 , X 2 , … ∼ p 0 i .…”

“…Let R be an ordered list of branches from to b = 1, the earliest branch. In the example in Figure 6, R = [4, 2, 1]. Finally, let t jj ′ be the length of the shortest path from leaf j to leaf j ′, the time from the most recent common ancestor to j plus the time to j ′.…”

EvoVGM: a Deep Variational Generative Model for Evolutionary Parameter Estimation

“…Moreover, VI is also used in building deep generative models [15,19]. However, contrary to Bayesian phylogenetic inference frameworks, most evolutionary-oriented deep generative models do not explicitly consider the underlying evolutionary dynamics of the biological sequences [1,22,26].…”