Approximate inference by Markov chains on union spaces

Welling, Max; Rosen-Zvi, Michal; Teh, Yee Whye

doi:10.1145/1015330.1015396

Cited by 121 publications

(165 citation statements)

References 3 publications

Supporting

Mentioning

164

Contrasting

Order By: Relevance

“…Hence, if the variational frequentist estimate is consistent, then the variational Bayes posterior converges to a Gaussian with a mean centred at the true model parameter. Furthermore, since variational Bayes rests on optimization, variational inference easily takes advantage of methods such as stochastic optimization (Robbins and Monro 1951, Kushner and Yin 1997) and distributed optimization (though some MCMC methods can also exploit these innovations (Welling and Teh 2011, Ahmed et al. 2012)).…”

Section: Statistical Regularizationmentioning

confidence: 99%

Solving inverse problems using data-driven models

et al. 2019

View full text Add to dashboard Cite

Recent research in inverse problems seeks to develop a mathematically coherent foundation for combining data-driven models, and in particular those based on deep learning, with domain-specific knowledge contained in physical–analytical models. The focus is on solving ill-posed inverse problems that are at the core of many challenging applications in the natural sciences, medicine and life sciences, as well as in engineering and industrial applications. This survey paper aims to give an account of some of the main contributions in data-driven inverse problems.

show abstract

Section: Statistical Regularizationmentioning

confidence: 99%

Solving inverse problems using data-driven models

et al. 2019

View full text Add to dashboard Cite

show abstract

“…We quantified the effect of circular filters on motif recognition further by comparing network architectures with and without circular filters for a variety of hyperparameter combinations. These included the number of positive training examples, L 2 -regularization strength and the amount of noise injected into parameter updates via SGLD (Welling and Teh, 2011). To investigate the effect of the weighted sum of activations that appears in the CNN with circular filters (Fig.…”

Section: Resultsmentioning

confidence: 99%

“…Model training. Models were trained by minimizing the cross-entropy between network outputs and sequence labels using either mini-batch stochastic gradient descent or Stochastic Gradient Langevin Dynamics (SGLD) (Welling and Teh, 2011), depending on the experiment. When SGLD was used, the magnitude of the noise injected into the gradients was scaled by a factor γ , resulting in γϵN(0,1) as injected noise, with ϵ as learning rate.…”

Section: Methodsmentioning

confidence: 99%

Neural networks with circular filters enable data efficient inference of sequence motifs

Blum

Kollmann

2019

Bioinformatics

View full text Add to dashboard Cite

Motivation Nucleic acids and proteins often have localized sequence motifs that enable highly specific interactions. Due to the biological relevance of sequence motifs, numerous inference methods have been developed. Recently, convolutional neural networks (CNNs) have achieved state of the art performance. These methods were able to learn transcription factor binding sites from ChIP-seq data, resulting in accurate predictions on test data. However, CNNs typically distribute learned motifs across multiple filters, making them difficult to interpret. Furthermore, networks trained on small datasets often do not generalize well to new sequences. Results Here we present circular filters, a novel convolutional architecture, that convolves sequences with circularly permutated variants of the same filter. We motivate circular filters by the observation that CNNs frequently learn filters that correspond to shifted and truncated variants of the true motif. Circular filters enable learning of full-length motifs and allow easy interpretation of the learned filters. We show that circular filters improve motif inference performance over a wide range of hyperparameters as well as sequence length. Furthermore, we show that CNNs with circular filters in most cases outperform conventional CNNs at inferring DNA binding sites from ChIP-seq data. Availability and implementation Code is available at https://github.com/christopherblum. Supplementary information Supplementary data are available at Bioinformatics online.

show abstract

“…It should be emphasized that we should not consider these methods as in a pure competition; instead, they can be used in complement with each other. For example, stochastic gradient Langevin dynamics (SGLD) [40] can be viewed as a combination of gradient descent and annealing, and in [41], it is mentioned that inclusion of the deterministic hill climber (discrete version of gradient descent) can lead to a substantial speedup in the PMBGA.…”

Section: Discussionmentioning

confidence: 99%

Using recurrent neural networks to optimize dynamical decoupling for quantum memory

August

2017

Phys. Rev. A

View full text Add to dashboard Cite

We utilize machine learning models which are based on recurrent neural networks to optimize dynamical decoupling (DD) sequences. DD is a relatively simple technique for suppressing the errors in quantum memory for certain noise models. In numerical simulations, we show that with minimum use of prior knowledge and starting from random sequences, the models are able to improve over time and eventually output DD-sequences with performance better than that of the well known DD-families. Furthermore, our algorithm is easy to implement in experiments to find solutions tailored to the specific hardware, as it treats the figure of merit as a black box.

show abstract

Approximate inference by Markov chains on union spaces

Cited by 121 publications

References 3 publications

Solving inverse problems using data-driven models

Solving inverse problems using data-driven models

Neural networks with circular filters enable data efficient inference of sequence motifs

Using recurrent neural networks to optimize dynamical decoupling for quantum memory

Contact Info

Product

Resources

About