FADNet: A Fast and Accurate Network for Disparity Estimation

Abstract

Deep neural networks (DNNs) have achieved great success in the area of computer vision. The disparity estimation problem tends to be addressed by DNNs which achieve much better prediction accuracy in stereo matching than traditional hand-crafted feature based methods. On one hand, however, the designed DNNs require significant memory and computation resources to accurately predict the disparity, especially for those 3D convolution based networks, which makes it difficult for deployment in real-time applications. On the other hand, existing computation-efficient networks lack expression capability in large-scale datasets so that they cannot make an accurate prediction in many scenarios. To this end, we propose an efficient and accurate deep network for disparity estimation named FADNet with three main features: 1) It exploits efficient 2D based correlation layers with stacked blocks to preserve fast computation; 2) It combines the residual structures to make the deeper model easier to learn; 3) It contains multi-scale predictions so as to exploit a multi-scale weight scheduling training technique to improve the accuracy. We conduct experiments to demonstrate the effectiveness of FADNet on two popular datasets, Scene Flow and KITTI 2015. Experimental results show that FADNet achieves state-of-the-art prediction accuracy, and runs at a significant order of magnitude faster speed than existing 3D models. The codes of FADNet are available at https://github.com/HKBU-HPML/FADNet.

Multi-Scale Cost Volumes Cascade Network for Stereo Matching

“…In particular, compared with Fast DS-CS [2] based on ADCensus and DispNetC [3] based on 1D Correlation, our MSCVNet obviously better than them in all evaluation metrics on KITTI2012, and only a little worse than Fast DS-CS in D1fg on KITTI2015, demonstrating that integrating ADCensus and 1D Correlation is effective. Our network can achieve better performance than FADNet [24] while using less time. We also visualize the results in Fig.…”

SRH-Net: Stacked Recurrent Hourglass Network for Stereo Matching

“…All metrics are evaluated in non-occluded (Noc) and all (All) regions, respectively. Here the 2DCoder methods include DispNetC [10], FADNet [7] and CRL [11], while GC-Net [8], PSMNet [9] and GA-Net [2] belong to 3DConv approaches. To present the results more intuitively, DispNetC and GC-Net are selected as the baselines of those two classes, respectively.…”

“…Then those costs and features are regressed to a disparity map in the process of decoding. Inspired by the classical encoder-decoder architecture, CRL [11] provides a two-stage pipeline in a cascade manner, while FADNet [7] constructs multi-scale disparity predictions by building two parallel encoder-decoder modules. Both of them exploit residual structures to make their models deeper and make initial disparities more smooth.…”

“…According to the usage of 3D convolutions in stereo matching, those networks are classified into two groups: encoder-decoder networks based on 2D convolutional operations (2DCoder) [6], [7] and explicit cost aggregation networks with 3D convolutional filters (3DConv) [8], [9]. The earliest 2DCoder method can date back to the DispNet [10], in which a 1D correlation layer is used to generate a 3D cost volume.…”

“…By concatenating with a feature map, this volume can be conducive to the disparity regression in the process of decoding. Based on the DispNet, residual structures are used in CRL [11] and FADNet [7] to implement a deep network and refine the disparity across multiple scales. These methods represent matching costs in unary values to finish aggregating a 3D volume by 2D convolutions.…”