Equivariant Deep Learning via Morphological and Linear Scale Space PDEs on the Space of Positions and Orientations

We present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs). In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer’s trainable weights. Formulating our PDEs on homogeneous spaces allows these networks to be designed with built-in symmetries such as rotation in addition to the standard translation equivariance of CNNs. Having all the desired symmetries included in the design obviates the need to include them by means of costly techniques such as data augmentation. We will discuss our PDE-based G-CNNs (PDE-G-CNNs) in a general homogeneous space setting while also going into the specifics of our primary case of interest: roto-translation equivariance. We solve the PDE of interest by a combination of linear group convolutions and nonlinear morphological group convolutions with analytic kernel approximations that we underpin with formal theorems. Our kernel approximations allow for fast GPU-implementation of the PDE-solvers; we release our implementation with this article in the form of the LieTorch extension to PyTorch, available at https://gitlab.com/bsmetsjr/lietorch. Just like for linear convolution, a morphological convolution is specified by a kernel that we train in our PDE-G-CNNs. In PDE-G-CNNs, we do not use non-linearities such as max/min-pooling and ReLUs as they are already subsumed by morphological convolutions. We present a set of experiments to demonstrate the strength of the proposed PDE-G-CNNs in increasing the performance of deep learning-based imaging applications with far fewer parameters than traditional CNNs.

show abstract

“…This paper is a substantially extended journal version of [31] presented at the SSVM 2021 conference.…”

Section: Introductionmentioning

confidence: 99%

PDE-Based Group Equivariant Convolutional Neural Networks

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is closely related to the idea of deep energies [37], where one chooses a variational energy as a loss function. Since we do not have such an energy available for EED inpainting, we resort to minimising the absolute residual of the associated Euler-Lagrange equation, which is given by (29). This guarantees that the trained architecture realises EED inpainting as efficiently as possible.…”

Section: Methodsmentioning

confidence: 99%

Connections Between Numerical Algorithms for PDEs and Neural Networks

et al. 2022

View full text Add to dashboard Cite

We investigate numerous structural connections between numerical algorithms for partial differential equations (PDEs) and neural architectures. Our goal is to transfer the rich set of mathematical foundations from the world of PDEs to neural networks. Besides structural insights, we provide concrete examples and experimental evaluations of the resulting architectures. Using the example of generalised nonlinear diffusion in 1D, we consider explicit schemes, acceleration strategies thereof, implicit schemes, and multigrid approaches. We connect these concepts to residual networks, recurrent neural networks, and U-net architectures. Our findings inspire a symmetric residual network design with provable stability guarantees and justify the effectiveness of skip connections in neural networks from a numerical perspective. Moreover, we present U-net architectures that implement multigrid techniques for learning efficient solutions of partial differential equation models, and motivate uncommon design choices such as trainable nonmonotone activation functions. Experimental evaluations show that the proposed architectures save half of the trainable parameters and can thus outperform standard ones with the same model complexity. Our considerations serve as a basis for explaining the success of popular neural architectures and provide a blueprint for developing new mathematically well-founded neural building blocks.

show abstract

“…This is remedied by Weiler et al [78,80] who make use of steerable filters [27] to design a larger set of oriented filters. Duits et al [22] go one step further by formulating all layers as solvers to parametrised PDEs. Similar ideas have been implemented with wavelets [67] and circular harmonics [83], and the group invariance concept has also been extended to higher dimensional data [14,57,79].…”

Section: Related Workmentioning

confidence: 99%

Designing rotationally invariant neural networks from PDEs and variational methods

et al. 2022

View full text Add to dashboard Cite

Partial differential equation models and their associated variational energy formulations are often rotationally invariant by design. This ensures that a rotation of the input results in a corresponding rotation of the output, which is desirable in applications such as image analysis. Convolutional neural networks (CNNs) do not share this property, and existing remedies are often complex. The goal of our paper is to investigate how diffusion and variational models achieve rotation invariance and transfer these ideas to neural networks. As a core novelty, we propose activation functions which couple network channels by combining information from several oriented filters. This guarantees rotation invariance within the basic building blocks of the networks while still allowing for directional filtering. The resulting neural architectures are inherently rotationally invariant. With only a few small filters, they can achieve the same invariance as existing techniques which require a fine-grained sampling of orientations. Our findings help to translate diffusion and variational models into mathematically well-founded network architectures and provide novel concepts for model-based CNN design.

show abstract

Equivariant Deep Learning via Morphological and Linear Scale Space PDEs on the Space of Positions and Orientations

Cited by 10 publications

References 12 publications

PDE-Based Group Equivariant Convolutional Neural Networks

PDE-Based Group Equivariant Convolutional Neural Networks

Connections Between Numerical Algorithms for PDEs and Neural Networks

Designing rotationally invariant neural networks from PDEs and variational methods

Contact Info

Product

Resources

About