Dense Feature Learning and Compact Cost Aggregation for Deep Stereo Matching

Abstract

Recently, Convolutional Neural Networks (CNN) based deep models have been successfully applied to the task of stereo matching. In this paper, we propose a novel deep stereo matching network based on the strategies of dense feature learning and compact cost aggregation, namely DFL-CCA-Net. It consists of three modules: Dense Feature Learning (DFL), Compact Cost Aggregation (CCA) and the disparity regression module. In DFL module, the CNN backbone with Dense Atrous Spatial Pyramid Pooling (DenseASPP) is employed to extract multi-scale deep feature maps of the given left and right images respectively. Then an initial 4D cost volume is obtained by concatenating left feature maps with their corresponding right feature maps across each disparity level. In the following CCA module, each initial 3D cost volume component (i.e., the component across the left or right image feature channel dimension) is aggregated into a more compact one by using the atrous convolution operation with different expansion rates. These updated 3D cost volume components are then fed into the disparity regression module, which consisting of a 3D CNN network with a stacked hourglass structure, to estimate the final disparity map. Comprehensive experimental results demonstrated on the Scene Flow, KITTI 2012 and KITTI 2015 datasets show that the 3D cost volume components obtained by the proposed DFL and CCA modules generally containing more multi-scale semantic information and thus can largely improve the final disparity regression accuracies. Compared with other deep stereo matching methods, DFL-CCA-Net achieves very competitive prediction accuracies especially in the reflective regions and regions containing detail information. INDEX TERMS Deep stereo matching, dense feature learning, compact cost aggregation

End-to-end information fusion method for transformer-based stereo matching

“…The stereo matching method takes a pair of rectified left and right stereo images as input. The goal of stereo matching is to determine the pixel displacement or disparity between corresponding points in the left and right images, providing crucial information for reconstructing the 3D structure of the scene [2]. Therefore, stereo matching is widely used in various fields, including 3D reconstruction [3,4], 3D registration [5], robotics [6], medicine [7], and autonomous driving [8].…”

“…This module enhances relationships between feature maps and generates more representative image feature maps. (2) The idea of criss-cross attention is introduced in the transformer implicit matching stage, leveraging criss-cross transformer to better aggregate cross-epipolar information from multiple paths. The proposed method can effectively fuse image information and improve matching accuracy in low-texture regions.…”