Decomposing neural networks as mappings of correlation functions

Fischer, Katinka; Alexandre, René,; Christian, Keup,; Layer, Moritz; Dahmen, David; Helias, Moritz

doi:10.1103/physrevresearch.4.043143

Cited by 7 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A separate work, [36], presents close-to-Gaussian NN processes including stationary Bayesian posteriors in the joint limit of large width and large data set, using 1/N as an expansion parameter. Moreover, the authors of [37] explore a correspondence between learning dynamics in the continuous time limit and early Universe cosmology, and [38] analyzes connected correlation functions propagating through NN.…”

Section: Learningmentioning

confidence: 99%

Neural network field theories: non-Gaussianity, actions, and locality

Demirtas,

Halverson,

Maiti

et al. 2024

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Both the path integral measure in field theory and ensembles of neural networks describe distributions over functions. When the central limit theorem can be applied in the infinite-width (infinite-$N$) limit, the ensemble of networks corresponds to a free field theory. Although an expansion in $1/N$ corresponds to interactions in the field theory, others, such as in a small breaking of the statistical independence of network parameters, can also lead to interacting theories. These other expansions can be advantageous over the $1/N$-expansion, for example by improved behavior with respect to the universal approximation theorem. Given the connected correlators of a field theory, one can systematically reconstruct the action order-by-order in the expansion parameter, using a new Feynman diagram prescription whose vertices are the connected correlators. This method is motivated by the Edgeworth expansion and allows one to derive actions for neural network field theories. Conversely, the correspondence allows one to engineer architectures realizing a given field theory by representing action deformations as deformations of neural network parameter densities. As an example, $\phi^4$ theory is realized as an infinite-$N$ neural network field theory.

show abstract

Section: Learningmentioning

confidence: 99%

Neural network field theories: non-Gaussianity, actions, and locality

Demirtas,

Halverson,

Maiti

et al. 2024

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…Our new approach emphasizes the role of information and information geometry in renormalization, which we now understand broadly as a mechanism for identifying equivalence classes of probability distributions that are indistinguishable as predictive models at a level of precision fixed by the amount of accessible data. This viewpoint allows us to draw a very sharp analogy between renormalization and aspects of data compression [36,[55][56][57][58], data generation [59][60][61], data classification [40,[62][63][64][65][66][67], dimensional reduction and model selection [33,34,68] commonly studied in data science and machine learning. We present this approach to renormalization through its relationship with Bayesian inference to highlight its information theoretic origin.…”

Section: Bayesian Renormalization and Information Geometrymentioning

confidence: 99%

Bayesian renormalization

Berman,

Klinger,

Stapleton

2023

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

In this note we present a fully information theoretic approach to renormalization inspired by Bayesian statistical inference, which we refer to as Bayesian Renormalization. The main insight of Bayesian Renormalization is that the Fisher metric defines a correlation length that plays the role of an emergent RG scale quantifying the distinguishability between nearby points in the space of probability distributions. This RG scale can be interpreted as a proxy for the maximum number of unique observations that can be made about a given system during a statistical inference experiment. The role of the Bayesian Renormalization scheme is subsequently to prepare an effective model for a given system up to a precision which is bounded by the aforementioned scale. In applications of Bayesian Renormalization to physical systems, the emergent information theoretic scale is naturally identified with the maximum energy that can be probed by current experimental apparatus, and thus Bayesian Renormalization coincides with ordinary renormalization. However, Bayesian Renormalization is sufficiently general to apply even in circumstances in which an immediate physical scale is absent, and thus provides an ideal approach to renormalization in data science contexts. To this end, we provide insight into how the Bayesian Renormalization scheme relates to existing methods for data compression and data generation such as the information bottleneck and the diffusion learning paradigm. We conclude by designing an explicit form of Bayesian Renormalization inspired by Wilson's momentum shell renormalization scheme in Quantum Field Theory. We apply this Bayesian Renormalization scheme to a simple Neural Network and verify the sense in which it organizes the parameters of the model according to a hierarchy of information theoretic importance.

show abstract

“…The probability mapping problem in deep feedforward networks can be represented as an iterative transformation of distributions, which is crucial for the recognition and representation of data information. Strengthening the correlation between information processing can provide an interpretable perspective for classification [15]. Deep convolutional neural networks have been widely used in many fields such as natural language processing and computer vision.…”

Section: Related Workmentioning

confidence: 99%

Civil Engineering Structural Damage Identification by Integrating Benchmark Numerical Model and PCNN Network

Qiu,

Zhang

2023

IEEE Access

View full text Add to dashboard Cite

Strengthening the real-time and accurate identification of civil structures is of great significance for ensuring the safety and service life of civil engineering projects. Therefore, in order to reduce the incidence of civil structural accidents in buildings and improve the safety, availability, and integrity of building structures, the study plans to adopt deep learning methods. In this study, the parallel Convolutional neural network covering one-dimensional and two-dimensional features is combined with the Benchmark numerical model to identify structural damage. This network structure can effectively utilize two parallel branches to extract response features at different scales and time domains, ensuring the coverage of damage feature recognition content to a certain extent. And the Benchmark numerical model can effectively improve the visualization of identification in the simulation of civil structures. By testing the fusion algorithm model, the results show that the network structure can effectively extract damage signal features, and its minimum classification loss value can approach 0.01; The maximum damage indicators on connecting beams, frame beams, and shear walls reached 0.472, 0.117, and 0.055, far higher than other comparative algorithms. The fusion algorithm has a recognition accuracy of over 85% for structural joint damage, showing good damage recognition performance. This fusion algorithm can effectively provide reference value and significance for the development of structural inspection and related risk prevention plans in civil engineering projects, and also provide new ideas and possibilities for relevant researchers to study the field of civil engineering.

show abstract

Decomposing neural networks as mappings of correlation functions

Cited by 7 publications

References 24 publications

Neural network field theories: non-Gaussianity, actions, and locality

Neural network field theories: non-Gaussianity, actions, and locality

Bayesian renormalization

Civil Engineering Structural Damage Identification by Integrating Benchmark Numerical Model and PCNN Network

Contact Info

Product

Resources

About