Roto-Translation Equivariant Convolutional Networks: Application to Histopathology Image Analysis

Abstract

Rotation-invariance is a desired property of machine-learning models for medical image analysis and in particular for computational pathology applications. We propose a framework to encode the geometric structure of the special Euclidean motion group SE(2) in convolutional networks to yield translation and rotation equivariance via the introduction of SE(2)group convolution layers. This structure enables models to learn feature representations with a discretized orientation dimension that guarantees that their outputs are invariant under a discrete set of rotations.Conventional approaches for rotation invariance rely mostly on data augmentation, but this does not guarantee the robustness of the output when the input is rotated. At that, trained conventional CNNs may require test-time rotation augmentation to reach their full capability.This study is focused on histopathology image analysis applications for which it is desirable that the arbitrary global orientation information of the imaged tissues is not captured by the machine learning models. The proposed framework is evaluated on three different histopathology image analysis tasks (mitosis detection, nuclei segmentation and tumor classification). We present a comparative analysis for each problem and show that consistent increase of performances can be achieved when using the proposed framework.

Dense Steerable Filter CNNs for Exploiting Rotational Symmetry in Histology Images

“…However, any other rotation may give interpolation artefacts and therefore may have negative implications for rotation-equivariance. Therefore, in line with Marcos et al [4] and Lafarge et al [5], for both the VF-CNN and standard G-CNN, we apply circular masking to the filters when using the groups C 8 and C 12 . However, this masking still leads to inevitable interpolation artefacts in the centre of the filter.…”

“…CPath is ripe ground for the utilisation of rotationequivariant models, yet most models fail to incorporate this prior knowledge into the CNN architectures. Inspired by recent developments in the study of rotation-equivariant CNNs [2], [3], [4], [5], we propose Dense Steerable Filter based CNNs (DSF-CNNs) that integrate steerable filters [6] with group convolution [2] and a densely connected framework [7] for superior performance. Each filter is defined as a linear combination of circular harmonic basis filters, enabling exact rotation and significantly reducing the number of parameters compared to standard filters.…”

“…However, this strategy requires images to be resampled on a hexagonal lattice, which is an additional overhead. Instead of using exact filter rotations, Bekkers et al [30] and Lafarge et al [5] applied G-CNNs to several medical imaging tasks by rotating filters with bilinear interpolation. Therefore, this method was not restricted to rotations by multiples of 90 • , but may introduce interpolation artefacts.…”

Neural network for the prediction of treatment response in Triple Negative Breast Cancer *

“…In contrast, histopathology images have a rotationally invariant content with no prior regarding scale or positioning of the relevant structures. However, rotational invariance can be imposed [26, 27], and in practice ImageNet based encodings are widely used and tend to perform very well.…”

A cortical-inspired sub-Riemannian model for Poggendorff-type visual illusions

“…We mention the large corpus of literature by Duits et al, see e.g., [28][29][30] and the state-of-the-art image inpainting and image recognition algorithms developed by Boscain, Gauthier, et al [31,32]. Some extensions of the CPS model geometry and its applications to other image processing problems can be found in [33][34][35][36][37][38][39][40].…”