Generalization capabilities of translationally equivariant neural networks

Bulusu, Srinath; Favoni, Matteo; Ipp, Andreas; Müller, David I.; Schuh, Daniel

doi:10.48550/arxiv.2103.14686

Cited by 3 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The target densities defined in Table I are all invariant under translations with appropriate boundary conditions, as discussed in Section II C. Previous works have shown that exactly incorporating known symmetries into machine learning models can accelerate their training and improve their final quality [86][87][88][89][90][91]. In the context of normalizing flows, ensuring that the model density is invariant under a symmetry group is achieved by choosing an invariant prior distribution and building transformation layers that are equivariant under the symmetry.…”

Section: B Building Blocksmentioning

confidence: 99%

Flow-based sampling for fermionic lattice field theories

Albergo,

Kanwar,

Racanière

et al. 2021

Preprint

View full text Add to dashboard Cite

Algorithms based on normalizing flows are emerging as promising machine learning approaches to sampling complicated probability distributions in a way that can be made asymptotically exact. In the context of lattice field theory, proof-of-principle studies have demonstrated the effectiveness of this approach for scalar theories, gauge theories, and statistical systems. This work develops approaches that enable flow-based sampling of theories with dynamical fermions, which is necessary for the technique to be applied to lattice field theory studies of the Standard Model of particle physics and many condensed matter systems. As a practical demonstration, these methods are applied to the sampling of field configurations for a two-dimensional theory of massless staggered fermions coupled to a scalar field via a Yukawa interaction.

show abstract

Section: B Building Blocksmentioning

confidence: 99%

Flow-based sampling for fermionic lattice field theories

Albergo,

Kanwar,

Racanière

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Our results prove that linear G-CNNs are biased towards low rank solutions in the Fourier regime, via regularization of the 2/L-Schatten norms over Fourier matrix coefficients (also a quasi-norm). Lastly, there is a line of work focusing on understanding the expressivity Cohen et al, 2019;Yarotsky, 2021) and generalization (Sannai & Imaizumi, 2019;Lyle et al, 2020;Bulusu et al, 2021;Elesedy & Zaidi, 2021) of equivariant networks but not specifically the effects of implicit regularization.…”

Section: Related Workmentioning

confidence: 99%

Implicit Bias of Linear Equivariant Networks

Lawrence¹,

Kristian²,

Dienes³

et al. 2021

Preprint

View full text Add to dashboard Cite

Group equivariant convolutional neural networks (G-CNNs) are generalizations of convolutional neural networks (CNNs) which excel in a wide range of scientific and technical applications by explicitly encoding group symmetries, such as rotations and permutations, in their architectures. Although the success of G-CNNs is driven by the explicit symmetry bias of their convolutional architecture, a recent line of work has proposed that the implicit bias of training algorithms on a particular parameterization (or architecture) is key to understanding generalization for overparameterized neural nets. In this context, we show that L-layer full-width linear G-CNNs trained via gradient descent in a binary classification task converge to solutions with low-rank Fourier matrix coefficients, regularized by the 2/L-Schatten matrix norm. Our work strictly generalizes previous analysis on the implicit bias of linear CNNs to linear G-CNNs over all finite groups, including the challenging setting of non-commutative symmetry groups (such as permutations). We validate our theorems via experiments on a variety of groups and empirically explore more realistic nonlinear networks, which locally capture similar regularization patterns 1 . Finally, we provide intuitive interpretations of our Fourier space implicit regularization results in real space via uncertainty principles.

show abstract

“…Machine learning (ML) techniques provide powerful algorithms building models based on the correlation pattern of data. Various approaches enhancing the lattice QCD calculations using ML have been explored in recent studies [1][2][3][4]. However, the ML training, which is the procedure of finding optimal model parameters describing the data, of most of the ML algorithms involve nontrivial optimization problems.…”

Section: Introductionmentioning

confidence: 99%

Prediction and compression of lattice QCD data using machine learning algorithms on quantum annealer

Yoon¹,

Chang²,

Kenyon³

et al. 2022

Proceedings of the 38th International Symposium on Lattice Field Theory — PoS(LATTICE2021)

View full text Add to dashboard Cite

We present regression and compression algorithms for lattice QCD data utilizing the efficient binary optimization ability of quantum annealers. In the regression algorithm, we encode the correlation between the input and output variables into a sparse coding machine learning algorithm. The trained correlation pattern is used to predict lattice QCD observables of unseen lattice configurations from other observables measured on the lattice. In the compression algorithm, we define a mapping from lattice QCD data of floating-point numbers to the binary coefficients that closely reconstruct the input data from a set of basis vectors. Since the reconstruction is not exact, the mapping defines a lossy compression, but, a reasonably small number of binary coefficients are able to reconstruct the input vector of lattice QCD data with the reconstruction error much smaller than the statistical fluctuation. In both applications, we use D-Wave quantum annealers to solve the NP-hard binary optimization problems of the machine learning algorithms.

show abstract

Generalization capabilities of translationally equivariant neural networks

Cited by 3 publications

References 39 publications

Flow-based sampling for fermionic lattice field theories

Flow-based sampling for fermionic lattice field theories

Implicit Bias of Linear Equivariant Networks

Prediction and compression of lattice QCD data using machine learning algorithms on quantum annealer

Contact Info

Product

Resources

About