Stereo Matching Using Multi-Level Cost Volume and Multi-Scale Feature Constancy

Liang, Zhengfa; Guo, Yulan; Feng, Yan; Chen, Wei; Qiao, Linbo; Zhou, Li; Zhang, Jianfeng; Liu, Hengzhu

doi:10.1109/tpami.2019.2928550

Cited by 98 publications

(55 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As can be seen in Table 12, the synthetic-data pretrained Ada-PSMNet outperforms all other state-of-the-art end-to-end disparity networks (EdgeStereo (Song et al 2020), CasStereo (Gu et al 2020), iResNet (Liang et al 2018), MCV-MFC (Liang et al 2019) and PSMNet (Chang and Chen 2018)) which are finetuned using ground-truth disparities in the Middlebury training set by a noteworthy margin. Our Ada-PSMNet achieves a remarkable 2-pixel error rate of 13.7% on the full-resolution test set, outperforming all other finetuned end-to-end stereo matching networks on the benchmark.…”

Section: Results On the Middlebury Benchmarkmentioning

confidence: 92%

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

et al. 2022

View full text Add to dashboard Cite

Recently, records on stereo matching benchmarks are constantly broken by end-to-end disparity networks. However, the domain adaptation ability of these deep models is quite limited. Addressing such problem, we present a novel domain-adaptive approach called AdaStereo that aims to align multi-level representations for deep stereo matching networks. Compared to previous methods, our AdaStereo realizes a more standard, complete and effective domain adaptation pipeline. Firstly, we propose a non-adversarial progressive color transfer algorithm for input image-level alignment. Secondly, we design an efficient parameter-free cost normalization layer for internal feature-level alignment. Lastly, a highly related auxiliary task, self-supervised occlusion-aware reconstruction is presented to narrow the gaps in output space. We perform intensive ablation studies and break-down comparisons to validate the effectiveness of each proposed module. With no extra inference overhead and only a slight increase in training complexity, our AdaStereo models achieve state-of-the-art cross-domain performance on multiple benchmarks, including KITTI, Middlebury, ETH3D and DrivingStereo, even outperforming some state-of-the-art disparity networks finetuned with target-domain ground-truths. Moreover, based on two additional evaluation metrics, the superiority of our domain-adaptive stereo matching pipeline is further uncovered from more perspectives. Finally, we demonstrate that our method is robust to various domain adaptation settings, and can be easily integrated into quick adaptation application scenarios and real-world deployments.

show abstract

Section: Results On the Middlebury Benchmarkmentioning

confidence: 92%

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In stereo matching tasks, the cost volume performs matching cost calculation, whose purpose is to measure the correlation between the pixels to be matched and the candidate pixels [20][21][22][23]. Whether two pixels are homonymy points, the matching cost can be calculated by the matching cost function.…”

Section: Cost Volumementioning

confidence: 99%

Group‐Based Atrous Convolution Stereo Matching Network

Zou

Fang

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Stereo matching is the key technology in stereo vision. Given a pair of rectified images, stereo matching determines correspondences between the pair images and estimate depth by obtaining disparity between corresponding pixels. The current work has shown that depth estimation from a stereo pair of images can be formulated as a supervised learning task with an end-to-end frame based on convolutional neural networks (CNNs). However, 3D CNN puts a great burden on memory storage and computation, which further leads to the significantly increased computation time. To alleviate this issue, atrous convolution was proposed to reduce the number of convolutional operations via a relatively sparse receptive field. However, this sparse receptive field makes it difficult to find reliable corresponding points in fuzzy areas, e.g., occluded areas and untextured areas, owing to the loss of rich contextual information. To address this problem, we propose the Group-based Atrous Convolution Spatial Pyramid Pooling (GASPP) to robustly segment objects at multiple scales with affordable computing resources. The main feature of the GASPP module is to set convolutional layers with continuous dilation rate in each group, so that it can reduce the impact of holes introduced by atrous convolution on network performance. Moreover, we introduce a tailored cascade cost volume in a pyramid form to reduce memory, so as to meet real-time performance. The group-based atrous convolution stereo matching network is evaluated on the street scene benchmark KITTI 2015 and Scene Flow and achieves state-of-the-art performance.

show abstract

“…The rapid expansion of affordable 3D sensors of diverse types including 3D scanners, RGB-D cameras (Kinect, Apple depth cameras…) as well as LIDARS [2] have contributed immensely to the development of computer vision where semantic and instance segmentation are key components. Data acquired by these 3D sensors have the advantage of providing size information and rich geometric profile ( [3], [4]).…”

Section: Introductionmentioning

confidence: 99%

Deep Learning Methods Applied to 3D Point Clouds Based Instance Segmentation: A Review

Burume¹,

Du²

2021

Preprint

View full text Add to dashboard Cite

Beyond semantic segmentation,3D instance segmentation(a process to delineate objects of interest and also classifying the objects into a set of categories) is gaining more and more interest among researchers since numerous computer vision applications need accurate segmentation processes(autonomous driving, indoor navigation, and even virtual or augmented reality systems…) This paper gives an overview and a technical comparison of the existing deep learning architectures in handling unstructured Euclidean data for the rapidly developing 3D instance segmentation. First, the authors divide the 3D point clouds based instance segmentation techniques into two major categories which are proposal based methods and proposal free methods. Then, they also introduce and compare the most used datasets with regard to 3D instance segmentation. Furthermore, they compare and analyze these techniques performance (speed, accuracy, response to noise…). Finally, this paper provides a review of the possible future directions of deep learning for 3D sensor-based information and provides insight into the most promising areas for prospective research.

show abstract

Stereo Matching Using Multi-Level Cost Volume and Multi-Scale Feature Constancy

Cited by 98 publications

References 45 publications

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

Group‐Based Atrous Convolution Stereo Matching Network

Deep Learning Methods Applied to 3D Point Clouds Based Instance Segmentation: A Review

Contact Info

Product

Resources

About