Dynamic Group Convolution for Accelerating Convolutional Neural Networks

Abstract

Replacing normal convolutions with group convolutions can significantly increase the computational efficiency of modern deep convolutional networks, which has been widely adopted in compact network architecture designs. However, existing group convolutions undermine the original network structures by cutting off some connections permanently resulting in significant accuracy degradation. In this paper, we propose dynamic group convolution (DGC) that adaptively selects which part of input channels to be connected within each group for individual samples on the fly. Specifically, we equip each group with a small feature selector to automatically select the most important input channels conditioned on the input images. Multiple groups can adaptively capture abundant and complementary visual/semantic features for each input image. The DGC preserves the original network structure and has similar computational efficiency as the conventional group convolution simultaneously. Extensive experiments on multiple image classification benchmarks including CIFAR-10, CIFAR-100 and ImageNet demonstrate its superiority over the existing group convolution techniques and dynamic execution methods 4 . The code is available at https://github.com/zhuogege1943/dgc.

Reweighted Dynamic Group Convolution

“…Let's firstly review the principle of the vanilla dynamic group convolution (DGC) [22]. The input and output feature maps of a certain layer can be defined as X ∈ ℝ H×W×I and Y ∈ ℝ H×W×O respectively, where H, W, I (O) are the height, width, and the number of channels.…”

“…It is worth noting that such group assignment is static, which limits the expressive ability. To alleviate this issue, a multi-path pruning structure with the self-attention is proposed in DGC [22] to dynamically assign groups. The SE module is used to assign saliency score V i g to different channels.…”

“…For instance, half of the channels will be pruned if φ = 0.5. In DGC [22], φ is set as a fixed value for all the convolution layers.…”

“…Some studies [20,21] utilized differentiable binary matrices and group convolutional layers to learn the mapping between groups and channels. Dynamic group convolution (DGC) [22] bought the channel selector for heads based on self-attention [23] and few-shot learning [24,25]. These channel selectors pick the optimal combination of channels for each group, thus complementing the semantic information among different groups.…”

“…A variant of DGC [22], namely reweighted dynamic group convolution (RDGC), is proposed in this paper. The key concept of RDGC is shown in Fig.…”