Exchangeable Generative Models with Flow Scans

Abstract

In this work, we develop a new approach to generative density estimation for exchangeable, non-i.i.d. data. The proposed framework, FlowScan, combines invertible flow transformations with a sorted scan to flexibly model the data while preserving exchangeability. Unlike most existing methods, FlowScan exploits the intradependencies within sets to learn both global and local structure. FlowScan represents the first approach that is able to apply sequential methods to exchangeable density estimation without resorting to averaging over all possible permutations. We achieve new state-of-the-art performance on point cloud and image set modeling.

Sampling usingSU(N)gauge equivariant flows

“…To preserve a symmetry in a flow-based sampling model, it is sufficient to sample from a prior distribution that is exactly invariant under the symmetry and transform the samples using an invertible transformation that is equivariant under the symmetry [41][42][43], meaning that symmetry transformations t commute with application of the function,…”

“…Permutation equivariance is required to ensure that the kernel acts only based on the spectrum, not the particular order of eigenvalues produced during diagonalization. Normalizing flows that are permutation equivariant have previously been investigated in the machinelearning community to learn densities over sets (such as point-clouds, objects in a 3D scene, particles in molecular dynamics, and particle tracks in collider events) [41][42][43][45][46][47][48][49][50][51][52]. Such approaches are directly applicable to kernels for U(N ) variables, because the eigenvalues can be transformed independently.…”

Computing Absolute Free Energy with Deep Generative Models

“…Furthermore, in addition to translational and rotational symmetry, these models must account for the permutation symmetry with respect to individual solvent molecules. It is worth noting that multiple studies [13,26,39,40] have introduced approaches for designing deep generative models that are invariant to permutations. Combining these approaches with our framework to compute the absolute free energy of biomolecular systems with explicit solvation would be an exciting direction for future studies.…”