Categorical representation learning and RG flow operators for algorithmic classifiers

Abstract

Following the earlier formalism of the categorical representation learning, we discuss the construction of the ``RG-flow-based categorifier''. Borrowing ideas from the theory of renormalization group flows (RG) in quantum field theory, holographic duality, and hyperbolic geometry and combining them with neural ODE techniques, we construct a new algorithmic natural language processing (NLP) architecture, called the RG-flow categorifier or for short the RG categorifier, which is capable of data classification and generation in all layers. We apply our algorithmic platform to biomedical data sets and show its performance in the field of sequence-to-function mapping. In particular, we apply the RG categorifier to particular genomic sequences of flu viruses and show how our technology is capable of extracting the information from given genomic sequences, finding their hidden symmetries and dominant features, classifying them, and using the trained data to make a stochastic prediction of new plausible generated sequences associated with a new set of viruses which could avoid the human immune system.

Machine learning renormalization group for statistical physics

“…• Deep learning and RG: several studies [55][56][57][58][59] have drawn parallels between deep neural networks and RG. Notably, they recognize that generation is the inverse process of renormalization [6,11]. Therefore, generative models can be used to implement data-driven RG.…”

“…Despite their effectiveness in extracting features of stable phases, they lack controlled accuracy in predicting universal properties of phase transitions, as they did not learn the RG equation or the RG monotone. • RG flow-based generative modeling: techniques such as neural-RG [2,6] and RG-Flow [10,11] embed RG transformations in multi-level flow-based generative models [60][61][62], applying deep learning methods to learn optimal RG transformations from model Hamiltonians by minimizing free energy. These methods are based on the invertible RG framework, which designs the local RG transformation as a bijective (invertible) deterministic map from spin configurations to relevant and irrelevant features.…”

“…The fluctuation of the A 1 representation z 1 0 mediates an equal amount of positive correlation ⟨x i x j ⟩ between every pair of visible spins, which is indeed the dominant correlation pattern in a ferromagnetic Ising model. However, to better match the teacher model p tch (x), the ⟨x 0 x 2 ⟩ and ⟨x 1 x 3 ⟩ correlations should be further weakened relative to others because there is no direct coupling between these diagonal spins in the original teacher model equation (11). This is where the E representation plays a role, as the z 2 0 and z 3 0 fluctuations will mediate some negative correlation in ⟨x 0 x 2 ⟩ and ⟨x 1 x 3 ⟩ due to the minus signs in equation (12), which helps to bring the student model's visible distribution p std (x) closer to that of the teacher model p tch (x).…”

“…Prior research has demonstrated that neural networks can learn to perform hierarchical feature extraction at the configuration level [1][2][3][4][5][6][7][8][9][10][11]. However, a more fascinating aspect of RG is its ability to quantitatively analyze the flow of physics theory in the parameter space at the model level [12,13].…”