Recalibrating Fully Convolutional Networks With Spatial and Channel “Squeeze and Excitation” Blocks

Roy, Abhijit Guha; Navab, Nassir; Wachinger, Christian

doi:10.1109/tmi.2018.2867261

Cited by 374 publications

(277 citation statements)

References 18 publications

Supporting

Mentioning

266

Contrasting

Unclassified

Order By: Relevance

“…This can be alternatively interpreted as a spatially adapted attention mechanism for each feature map; unlike [15], where a single attention map is generated. Related to our work is the block proposed by Roy et al [17]. But, channel and spatial recalibration are considered separately, while we learn them jointly.…”

Section: A Motivation and Contributionsmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Feature Recombination and Recalibration for Semantic Segmentation With Fully Convolutional Networks

Pereira

Pinto

Amorim

et al. 2019

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Fully Convolutional Networks have been achieving remarkable results in image semantic segmentation, while being efficient. Such efficiency results from the capability of segmenting several voxels in a single forward pass. So, there is a direct spatial correspondence between a unit in a feature map and the voxel in the same location. In a convolutional layer, the kernel spans over all channels and extracts information from them. We observe that linear recombination of feature maps by increasing the number of channels followed by compression may enhance their discriminative power. Moreover, not all feature maps have the same relevance for the classes being predicted. In order to learn the inter-channel relationships and recalibrate the channels to suppress the less relevant ones, Squeeze and Excitation blocks were proposed in the context of image classification with Convolutional Neural Networks. However, this is not well adapted for segmentation with Fully Convolutional Networks since they segment several objects simultaneously, hence a feature map may contain relevant information only in some locations. In this paper, we propose recombination of features and a spatially adaptive recalibration block that is adapted for semantic segmentation with Fully Convolutional Networks -the SegSE block. Feature maps are recalibrated by considering the cross-channel information together with spatial relevance. Experimental results indicate that Recombination and Recalibration improve the results of a competitive baseline, and generalize across three different problems: brain tumor segmentation, stroke penumbra estimation, and ischemic stroke lesion outcome prediction. The obtained results are competitive or outperform the state of the art in the three applications.

show abstract

Section: A Motivation and Contributionsmentioning

confidence: 99%

“…However, this approach scales the same regions across all feature maps. Roy et al [17] learn attention at the channel and spatial levels, but a single spatial attention map is inferred for all feature maps.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Feature Recombination and Recalibration for Semantic Segmentation With Fully Convolutional Networks

Pereira

Pinto

Amorim

et al. 2019

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…To our knowledge, attention modules in deep learning either compute the entire self-attention matrix on a low dimensional input or use a local attention mechanism that can be seen as a strong approximation of the non-local self-attention formulation. Specifically in the medical imaging context, previous works [12,15,11] implicitly used a simplification of (2) with a diagonal self-attention matrix. This solution can be applied to large images since it scales linearly with the number of voxels but does not help to capture contextual information.…”

Section: Methodsmentioning

confidence: 99%

Permutohedral Attention Module for Efficient Non-local Neural Networks

Joutard

Dorent

Isaac

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Medical image processing tasks such as segmentation often require capturing non-local information. As organs, bones, and tissues share common characteristics such as intensity, shape, and texture, the contextual information plays a critical role in correctly labeling them. Segmentation and labeling is now typically done with convolutional neural networks (CNNs) but the context of the CNN is limited by the receptive field which itself is limited by memory requirements and other properties. In this paper, we propose a new attention module, that we call Permutohedral Attention Module (PAM), to efficiently capture nonlocal characteristics of the image. The proposed method is both memory and computationally efficient. We provide a GPU implementation of this module suitable for 3D medical imaging problems. We demonstrate the efficiency and scalability of our module with the challenging task of vertebrae segmentation and labeling where context plays a crucial role because of the very similar appearance of different vertebrae.

show abstract

“…The U-Net has, since its creation, been subject to modifications and improvements, with the addition of more complex convolutional blocks [74] or squeeze-and-excite methods. [75] While the SSD network structure was well suited for object identification tasks, such as the automated identification of cells, the architecture of the U-Net is particularly optimized for the partitioning of images based on image features such as shapes and edges. This makes the U-Net a natural choice for image segmentation tasks.…”

Section: Deep Learning Finds the Contours Of Cells With Segmentation mentioning

confidence: 99%

Biomedical Image Processing with Containers and Deep Learning: An Automated Analysis Pipeline

González

Evans

2019

BioEssays

View full text Add to dashboard Cite

Here, a streamlined, scalable, laboratory approach is discussed that enables medium‐to‐large dataset analysis. The presented approach combines data management, artificial intelligence, containerization, cluster orchestration, and quality control in a unified analytic pipeline. The unique combination of these individual building blocks creates a new and powerful analysis approach that can readily be applied to medium‐to‐large datasets by researchers to accelerate the pace of research. The proposed framework is applied to a project that counts the number of plasmonic nanoparticles bound to peripheral blood mononuclear cells in dark‐field microscopy images. By using the techniques presented in this article, the images are automatically processed overnight, without user interaction, streamlining the path from experiment to conclusions.

show abstract

Recalibrating Fully Convolutional Networks With Spatial and Channel “Squeeze and Excitation” Blocks

Cited by 374 publications

References 18 publications

Adaptive Feature Recombination and Recalibration for Semantic Segmentation With Fully Convolutional Networks

Adaptive Feature Recombination and Recalibration for Semantic Segmentation With Fully Convolutional Networks

Permutohedral Attention Module for Efficient Non-local Neural Networks

Biomedical Image Processing with Containers and Deep Learning: An Automated Analysis Pipeline

Contact Info

Product

Resources

About