Truly Shift-Equivariant Convolutional Neural Networks with Adaptive Polyphase Upsampling

Chaman, Anadi

doi:10.1109/ieeeconf53345.2021.9723377

Cited by 5 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In practice, equivariance may be broken by incorporating additional operations such as downsampling and upsampling, although this issue can be overcome by using suitable replacements of these operations. This can be done for instance using adaptive polyphase upsampling and downsampling, and was applied to some Fourier-based computational imaging tasks in [25]. It is worth noting that even so, edge effects that arise as a result of bounded image domains will always prevent exact translational equivariance from holding.…”

Section: Equivariance By Designmentioning

confidence: 99%

Imaging With Equivariant Deep Learning: From unrolled network design to fully unsupervised learning

Davies

Ehrhardt

et al. 2023

IEEE Signal Process. Mag.

View full text Add to dashboard Cite

From early image processing to modern computational imaging, successful models and algorithms have relied on a fundamental property of natural signals: symmetry. Here symmetry refers to the invariance property of signal sets to transformations such as translation, rotation or scaling. Symmetry can also be incorporated into deep neural networks in the form of equivariance, allowing for more data-efficient learning. While there has been important advances in the design of end-to-end equivariant networks for image classification in recent years, computational imaging introduces unique challenges for equivariant network solutions since we typically only observe the image through some noisy ill-conditioned forward operator that itself may not be equivariant. We review the emerging field of equivariant imaging and show how it can provide improved generalization and new imaging opportunities. Along the way we show the interplay between the acquisition physics and group actions and links to iterative reconstruction, blind compressed sensing and self-supervised learning.

show abstract

Section: Equivariance By Designmentioning

confidence: 99%

Imaging With Equivariant Deep Learning: From unrolled network design to fully unsupervised learning

Davies

Ehrhardt

et al. 2023

IEEE Signal Process. Mag.

View full text Add to dashboard Cite

show abstract

“…Machine learning algorithms employed to predict properties such as potential energy and atomic forces must yield consistent results, regardless of the molecule's rotational pose or ordering. To address this challenge, researchers have developed group-equivariant neural networks that preserve these symmetries [5,[7][8][9][10]. In a group-equivariant network, symmetry operations on the data, including rotations of pictures and molecules, and permutations of the labels of each particle, commute with the network's layers, ensuring that the same physical property is predicted irrespective of the input's orientation.…”

Section: Introductionmentioning

confidence: 99%

Unifying O(3) equivariant neural networks design with tensor-network formalism

Li,

Pengmei,

Zheng

et al. 2024

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Many learning tasks, including learning potential energy surfaces from ab initio calculations, involve global spatial symmetries and permutational symmetry between atoms or general particles. Equivariant graph neural networks are a standard approach to such problems, with one of the most successful methods employing tensor products between various tensors that transform under the spatial group. However, as the number of different tensors and the complexity of relationships between them increase, maintaining parsimony and equivariance becomes increasingly challenging. In this paper, we propose using fusion diagrams, a technique widely employed in simulating SU($2$)-symmetric quantum many-body problems, to design new equivariant components for equivariant neural networks. This results in a diagrammatic approach to constructing novel neural network architectures. When applied to particles within a given local neighborhood, the resulting components, which we term "fusion blocks," serve as universal approximators of any continuous equivariant function defined on the neighborhood. We incorporate a fusion block into pre-existing equivariant architectures (Cormorant and MACE), leading to improved performance with fewer parameters on a range of challenging chemical problems. Furthermore, we apply group-equivariant neural networks to study non-adiabatic molecular dynamics of stilbene cis-trans isomerization and water adsorption cycles in porous materials, such as metal-organic frameworks. Our approach, which combines tensor networks with equivariant neural networks, suggests a potentially fruitful direction for designing more expressive equivariant neural networks.

show abstract

“…However, recent research has reported that CNNs can significantly degrade classification accuracy even with a single pixel translation [11], [12], and many shift-invariant feature representation methods have been proposed [13], [14]. Manfredi et al [15] reported performance degradation in object detection with small translations.…”

Section: Introductionmentioning

confidence: 99%

Improved Head and Data Augmentation to Reduce Artifacts at Grid Boundaries in Object Detection

UCHINOURA,

KURITA

2024

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

We investigated the influence of horizontal shifts of the input images for one stage object detection method. We found that the object detector class scores drop when the target object center is at the grid boundary. Many approaches have focused on reducing the aliasing effect of down-sampling to achieve shift-invariance. However, down-sampling does not completely solve this problem at the grid boundary; it is necessary to suppress the dispersion of features in pixels close to the grid boundary into adjacent grid cells. Therefore, this paper proposes two approaches focused on the grid boundary to improve this weak point of current object detection methods. One is the Sub-Grid Feature Extraction Module, in which the sub-grid features are added to the input of the classification head. The other is Grid-Aware Data Augmentation, where augmented data are generated by the grid-level shifts and are used in training. The effectiveness of the proposed approaches is demonstrated using the COCO validation set after applying the proposed method to the FCOS architecture.

show abstract

Truly Shift-Equivariant Convolutional Neural Networks with Adaptive Polyphase Upsampling

Cited by 5 publications

References 8 publications

Imaging With Equivariant Deep Learning: From unrolled network design to fully unsupervised learning

Imaging With Equivariant Deep Learning: From unrolled network design to fully unsupervised learning

Unifying O(3) equivariant neural networks design with tensor-network formalism

Improved Head and Data Augmentation to Reduce Artifacts at Grid Boundaries in Object Detection

Contact Info

Product

Resources

About