SDC – Stacked Dilated Convolution: A Unified Descriptor Network for Dense Matching Tasks

Abstract

Dense pixel matching is important for many computer vision tasks such as disparity and flow estimation. We present a robust, unified descriptor network that considers a large context region with high spatial variance. Our network has a very large receptive field and avoids striding layers to maintain spatial resolution. These properties are achieved by creating a novel neural network layer that consists of multiple, parallel, stacked dilated convolutions (SDC). Several of these layers are combined to form our SDC descriptor network. In our experiments, we show that our SDC features outperform state-of-the-art feature descriptors in terms of accuracy and robustness. In addition, we demonstrate the superior performance of SDC in state-of-the-art stereo matching, optical flow and scene flow algorithms on several famous public benchmarks. SDC Layer 1Input: M×N×3 Convolutions: 4 Sizes: [5, 5, 5, 5] Kernels: [16, 16, 16, 16] Dilations: [1, 2, 3, 4] Output: M×N×64 Color ImageFeature Map SDC Layer 2Input: M×N×64 Figure 2: Our SDC feature network. It consists of 5 SDC blocks with varying number of output channels. The final feature vectors are normalized to unit range pixel-wise. Convolution Dilation rate: 1 Convolution Dilation rate: 2 Convolution Dilation rate: 3 Convolution Dilation rate: 4 SDC Layer Parallel convolutions: 4

An Empirical Evaluation Study on the Training of SDC Features for Dense Pixel Matching

“…The proposed architecture of [27] consists of five such stacked dilated convolution layers, each with 4 parallel convolutions with 5 × 5 kernels and dilation rates d = 1, 2, 3, 4. This setup yields a receptive field of 81 pixels with a dense feature prediction for every input pixel.…”

“…In [27], a triplet training strategy was applied, where a reference image patch along with the corresponding and a non-corresponding patch are sampled randomly. SDC was [27]. The color gives the respective sensitivity to the RGB color channels of the input images.…”

“…positive distance is below a threshold and the negative distance is higher than the margin (cf. the thresholded hinge embedding loss in [27]). The expected behavior of training with hard mining is threefold.…”

“…Our use case is embedded in the context of environmental perception for automotive applications. In detail, the network under consideration is the recently presented SDC network [27] that was designed for image description to aid dense matching tasks, like in optical flow or stereo disparity estimation. Matching is a mid-level computer vision task that can be used to reconstruct geometry and estimate motions and therefore it builds the foundation for high-level perception and planning tasks which are required for advanced driver assistance systems and autonomous vehicles.…”

“…Matching Tasks. SDC [27] was presented as a generic feature descriptor that can be used for any dense matching task, e.g. stereo, optical flow, or scene flow matching.…”