Sparse generative modeling via parameter reduction of Boltzmann machines: Application to protein-sequence families

Barrat-Charlaix, Pierre; Muntoni, Anna Paola; Shimagaki, Kai; Weigt, Martin; Zamponi, Francesco

doi:10.1103/physreve.104.024407

Cited by 18 publications

(34 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another interesting observable that can be used to assess the generative power of the Boltzmann machine, is the three-site connected correlation which is not fitted during the training but, as shown in [10,11], provides an interesting measure of the generative capability of the model. adabmDCA does not compute all possible third order connected correlations because this would be computationally heavy.…”

Section: Convergence Criterion and Quality Controlmentioning

confidence: 99%

“…for which all couplings J are in principle different from 0) produces an over-parametrization of the unknown distribution as signaled by a large amount of noisy and negligible coupling parameters [11]. A practical way to control this behavior is to impose a sparsity prior over the coupling matrices: the two most used priors are the so called ℓ 1 and ℓ 2 regularizations, which force the inferred couplings to minimize the associated ℓ 1 and ℓ 2 norms multiplied by a tunable parameter that sets the regularization strength.…”

Section: An Introduction To Boltzmann Learning Of Biological Modelsmentioning

confidence: 99%

“…A complementary approach consists in setting a priori a probable topology suggested by the mutual information between all pairs of residues [22]. Here, as discussed in the following section, we will follow an information-based decimation protocol originally proposed in [11].…”

Section: An Introduction To Boltzmann Learning Of Biological Modelsmentioning

confidence: 99%

“…In a nutshell, the DCA inference strategy provides a simple and informative interpretation of the inferred set of model parameters in terms of remarkably accurate contact map prediction. Among the different DCA implementations, Boltzmann machine learning [8][9][10][11] turns out to be one of the most efficient in terms of: (1) the accuracy of structural predictions in its direct use, or as input of more complex deep learning supervised strategies [12][13][14][15]; (2) the effectiveness to generate artificially-designed sequences that fold similarly to their natural counterparts [16]; (3) the ability to predict mutational effects [17,18]. However, the quality of the Boltzmann machine deeply depends on the quality of the learning, which is intrinsically linked to the way the model observables are computed within the training, usually employing a Monte Carlo Markov Chain (MCMC).…”

mentioning

confidence: 99%

“…The code implements the socalled Boltzmann machine learning algorithm [19] by performing a gradient ascent of the a posteriori probability of the model given the input data. At variance with other implementations, adabmDCA copes with both protein and RNA data, and encompasses the ability of pruning redundant parameters (as described in [11]). Finally, it provides a standard equilibrium learning and also a non-equilibrium learning, based on the contrastive divergence technique [20], which is more suitable for structured data.…”

mentioning

confidence: 99%

See 4 more Smart Citations

adabmDCA: adaptive Boltzmann machine learning for biological sequences

et al. 2021

Self Cite

View full text Add to dashboard Cite

Background Boltzmann machines are energy-based models that have been shown to provide an accurate statistical description of domains of evolutionary-related protein and RNA families. They are parametrized in terms of local biases accounting for residue conservation, and pairwise terms to model epistatic coevolution between residues. From the model parameters, it is possible to extract an accurate prediction of the three-dimensional contact map of the target domain. More recently, the accuracy of these models has been also assessed in terms of their ability in predicting mutational effects and generating in silico functional sequences. Results Our adaptive implementation of Boltzmann machine learning, , can be generally applied to both protein and RNA families and accomplishes several learning set-ups, depending on the complexity of the input data and on the user requirements. The code is fully available at https://github.com/anna-pa-m/adabmDCA. As an example, we have performed the learning of three Boltzmann machines modeling the Kunitz and Beta-lactamase2 protein domains and TPP-riboswitch RNA domain. Conclusions The models learned by are comparable to those obtained by state-of-the-art techniques for this task, in terms of the quality of the inferred contact map as well as of the synthetically generated sequences. In addition, the code implements both equilibrium and out-of-equilibrium learning, which allows for an accurate and lossless training when the equilibrium one is prohibitive in terms of computational time, and allows for pruning irrelevant parameters using an information-based criterion.

show abstract

Section: Convergence Criterion and Quality Controlmentioning

confidence: 99%

Section: An Introduction To Boltzmann Learning Of Biological Modelsmentioning

confidence: 99%

Section: An Introduction To Boltzmann Learning Of Biological Modelsmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

adabmDCA: adaptive Boltzmann machine learning for biological sequences

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

Emergent time scales of epistasis in protein evolution

Di Bari,

Bisardi,

Cotogno

et al. 2024

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

We introduce a data-driven epistatic model of protein evolution, capable of generating evolutionary trajectories spanning very different time scales reaching from individual mutations to diverged homologs. Our in silico evolution encompasses random nucleotide mutations, insertions and deletions, and models selection using a fitness landscape, which is inferred via a generative probabilistic model for protein families. We show that the proposed framework accurately reproduces the sequence statistics of both short-time (experimental) and long-time (natural) protein evolution, suggesting applicability also to relatively data-poor intermediate evolutionary time scales, which are currently inaccessible to evolution experiments. Our model uncovers a highly collective nature of epistasis, gradually changing the fitness effect of mutations in a diverging sequence context, rather than acting via strong interactions between individual mutations. This collective nature triggers the emergence of a long evolutionary time scale, separating fast mutational processes inside a given sequence context, from the slow evolution of the context itself. The model quantitatively reproduces epistatic phenomena such as contingency and entrenchment, as well as the loss of predictability in protein evolution observed in deep mutational scanning experiments of distant homologs. It thereby deepens our understanding of the interplay between mutation and selection in shaping protein diversity and functions, allows one to statistically forecast evolution, and challenges the prevailing independent-site models of protein evolution, which are unable to capture the fundamental importance of epistasis.

show abstract

Optimal regularizations for data generation with probabilistic graphical models

Fanthomme¹,

Rizzato²,

Cocco³

et al. 2022

J. Stat. Mech.

View full text Add to dashboard Cite

Understanding the role of regularization is a central question in statistical inference. Empirically, well-chosen regularization schemes often dramatically improve the quality of the inferred models by avoiding overfitting of the training data. We consider here the particular case of L 2 regularization in the maximum a posteriori (MAP) inference of generative pairwise graphical models. Based on analytical calculations on Gaussian multivariate distributions and numerical experiments on Gaussian and Potts models we study the likelihoods of the training, test, and ‘generated data’ (with the inferred models) sets as functions of the regularization strengths. We show in particular that, at its maximum, the test likelihood and the ‘generated’ likelihood, which quantifies the quality of the generated samples, have remarkably close values. The optimal value for the regularization strength is found to be approximately equal to the inverse sum of the squared couplings incoming on sites on the underlying network of interactions. Our results seem to be robust against changes in the structure of the ground-truth underlying interactions that generated the data, when small fluctuations of the posterior distribution around the MAP estimator are taken into account, and when L 1 regularization is considered (instead of L 2). Connections with empirical works on protein models learned from homologous sequences are discussed.

show abstract

Sparse generative modeling via parameter reduction of Boltzmann machines: Application to protein-sequence families

Cited by 18 publications

References 45 publications

adabmDCA: adaptive Boltzmann machine learning for biological sequences

adabmDCA: adaptive Boltzmann machine learning for biological sequences

Emergent time scales of epistasis in protein evolution

Optimal regularizations for data generation with probabilistic graphical models

Contact Info

Product

Resources

About