Active Convolution: Learning the Shape of Convolution for Image Classification

Abstract

In recent years, deep learning has achieved great success in many computer vision applications. Convolutional neural networks (CNNs) have lately emerged as a major approach to image classification. Most research on CNNs thus far has focused on developing architectures such as the Inception and residual networks. The convolution layer is the core of the CNN, but few studies have addressed the convolution unit itself. In this paper, we introduce a convolution unit called the active convolution unit (ACU). A new convolution has no fixed shape, because of which we can define any form of convolution. Its shape can be learned through backpropagation during training. Our proposed unit has a few advantages. First, the ACU is a generalization of convolution; it can define not only all conventional convolutions, but also convolutions with fractional pixel coordinates. We can freely change the shape of the convolution, which provides greater freedom to form CNN structures. Second, the shape of the convolution is learned while training and there is no need to tune it by hand. Third, the ACU can learn better than a conventional unit, where we obtained the improvement simply by changing the conventional convolution to an ACU. We tested our proposed method on plain and residual networks, and the results showed significant improvement using our method on various datasets and architectures in comparison with the baseline.

Selective Kernel Networks

“…Dynamic Filter [20] can only adaptively modify the parameters of filters, without the adjustment of kernel size. Active Convolution [19] augments the sampling locations in the convolution with offsets. These offsets are learned end-to-end but become static after training, while in SKNet the RF sizes of neurons can adaptively change during inference.…”

Interaction-And-Aggregation Network for Person Re-Identification

“…Spatial Transform Network [18] warped the feature map via a global parametric transformation. The works [19,9] augmented the sampling locations in the convolution with offsets and learn the offsets via back-propagation end-to-end.…”

Local Relation Networks for Image Recognition

“…The most common parameterization method is to directly learn the aggregation weights ω [18]. There are also some methods that learn a meta network {θ} on input features to generate adaptive aggregation weights [15] or an adaptive aggregation scope across spatial positions [6], or learn a fixed prior about spatial aggregation scope (Ω) [14].…”

“…The "aggregation weight" column covers three aspects: how aggregation weights are computed from parameterized weights ("computation" sub-column); inclusion of geometric priors ("geo." sub-column); type of computation ("type" sub-column regular ω all/one/no local ω top-down group [17,28] ω group/one/no local ω top-down depthwise [5,12] ω one/one/no local ω top-down dilated [4,29] ω all/one/no atrous ω top-down active [14] ω, Ω all/one/no Ω ω top-down local connected [24] ω all/one/no local ω top-down dynamic filters [15] θ all/one/no local f θ (x p ) top-down deformable [6,32] ω, θ all/one/no…”