Graph-Based Global Reasoning Networks

Abstract

Globally modeling and reasoning over relations between regions can be beneficial for many computer vision tasks on both images and videos. Convolutional Neural Networks (CNNs) excel at modeling local relations by convolution operations, but they are typically inefficient at capturing global relations between distant regions and require stacking multiple convolution layers. In this work, we propose a new approach for reasoning globally in which a set of features are globally aggregated over the coordinate space and then projected to an interaction space where relational reasoning can be efficiently computed. After reasoning, relation-aware features are distributed back to the original coordinate space for down-stream tasks. We further present a highly efficient instantiation of the proposed approach and introduce the Global Reasoning unit (GloRe unit) that implements the coordinate-interaction space mapping by weighted global pooling and weighted broadcasting, and the relation reasoning via graph convolution on a small graph in interaction space. The proposed GloRe unit is lightweight, end-to-end trainable and can be easily plugged into existing CNNs for a wide range of tasks. Extensive experiments show our GloRe unit can consistently boost the performance of state-of-the-art backbone architectures, including ResNet [15,16], ResNeXt [33], SE-Net [18] and DPN [9], for both 2D and 3D CNNs, on image classification, semantic segmentation and video action recognition task.

Video Classification With Channel-Separated Convolutional Networks

“…Our ip-CSN-152, pre-trained on Sports1M outperforms I3D [3], R(2+1)D [32], and S3D-G [40] by 8.1%, 4.9%, and 4.5%, respectively. It also outperforms recent work: A 2 -Net [4] by 4.6%, Globalreasoning networks [6] by 3.1%. We note that our ip-CSN-152 achieves higher accuracy than both I3D with Non-local Networks (NL) [37] and SlowFast [10] (+1.5% and +0.3%) while being also faster (3.3x and 2x, respectively).…”

Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution

“…In the above experiment, we have compared and shown that OctConv is complementary with a set of state-of-the-art CNNs [16,17,47,22,18,34,19]. In this part, we compare OctConv with MG-Conv [25], GloRe [8], Elastic [43] and bL-Net [4] which share a similar idea as our method. Seven groups of results are shown in Table 4.…”

Image Super-Resolution via Attention Based Back Projection Networks

“…[15] proposed an efficient attention computation mechanism called Criss-Cross Network for semantic segmentation. [5] used the idea of bilateral filter to learn robust weighting model for object recognition. Besides, "attention" has also been proposed for image super-resolution and shown its great potential.…”