Semantic Stereo Matching With Pyramid Cost Volumes

Wu, Zhenyao; Wu, Xinyi; Zhang, Xiaoping; Wang, Song; Ju, Lili

doi:10.1109/iccv.2019.00758

Cited by 95 publications

(45 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, researchers have begun to use multitask learning [32] or two-branch structure to improve the network's ability to match challenging areas. SSPCVNet [33] corrects object boundary disparity values by adding a semantic segmentation sub-network in the cost aggregation network. Similarly, EdgeStereo and EdgeStereo-V2 [18] design an edge detection sub-network to correct the disparity values of challenging areas.…”

Section: Cost Aggregation Networkmentioning

confidence: 99%

Bidirectional Stereo Matching Network With Double Cost Volumes

2021

View full text Add to dashboard Cite

In stereo matching, the high-quality cost volume is the key to improve the matching accuracy. Current stereo matching networks only use traditional methods or neural networks to generate one or more cost volumes. They do not consider combining different matching cost computation methods to improve the quality of cost volume. Therefore, we propose BSDCNet, a real-time stereo matching network consisting of two main modules: Double Matching Cost Computation and Bidirectional Cost Aggregation Network. The Double Matching Cost Computation module generates two different cost volumes through traditional methods and neural networks. The Bidirectional Cost Aggregation Network is a two-branch structure, which can aggregate the above two cost volumes with different network depths. Finally, we also design a feature fusion module(FFM) to fuse the two-branch features and use the result for disparity estimation. Our network only uses 3D cost volumes and two-dimensional convolutions. Thus it is much faster than state-of-the-art stereo networks (e.g., 36× than GC-Net, 16× than PSMNet, and 72× than GA-Net). Meanwhile, according to the KITTI official website, our network is more accurate than other fast stereo networks (e.g., Fast DS-CS, RTSNet, and DispNetC), demonstrating that our network can generate a real-time and accurate stereo matching result.

show abstract

Section: Cost Aggregation Networkmentioning

confidence: 99%

Bidirectional Stereo Matching Network With Double Cost Volumes

2021

View full text Add to dashboard Cite

show abstract

“…Based on the linear model of the cyclic neural network, SDR [ 28 ] refines the disparity plane by learning the correlation matrix between adjacent pixels and using global and local two-level optimization. SSPCV-NET [ 29 ] improves the aggregation cost by using multi-scale feature information to form a pyramid cost in the end-to-end neural network. With the development of self-supervised learning, PV Stereo [ 30 ] builds multi-scale cost measurements and updates disparity estimation at high resolution by using cyclic units and generates reliable semi-density disparity images that supervise training and perform self-supervised stereo matching.…”

Section: Related Workmentioning

confidence: 99%

A Fast Stereo Matching Network with Multi-Cross Attention

Wei

Zhu

et al. 2021

Sensors

View full text Add to dashboard Cite

Stereo matching networks based on deep learning are widely developed and can obtain excellent disparity estimation. We present a new end-to-end fast deep learning stereo matching network in this work that aims to determine the corresponding disparity from two stereo image pairs. We extract the characteristics of the low-resolution feature images using the stacked hourglass structure feature extractor and build a multi-level detailed cost volume. We also use the edge of the left image to guide disparity optimization and sub-sample with the low-resolution data, ensuring excellent accuracy and speed at the same time. Furthermore, we design a multi-cross attention model for binocular stereo matching to improve the matching accuracy and achieve end-to-end disparity regression effectively. We evaluate our network on Scene Flow, KITTI2012, and KITTI2015 datasets, and the experimental results show that the speed and accuracy of our method are excellent.

show abstract

“…When a single handheld RGB camera flies over a static scene, a detailed texture depth map of selected key frames is estimated to generate a patchwork of surfaces with millions of vertices. Aiming at the 3D structure restoration process in real-time 3D reconstruction problems, they proposed a Cost Volume-based 3D structure Recovery method [ 18 , 19 ]. Yang et al developed a real-time 3D reconstruction system suitable for UAVs on Jetson TX2 [ 20 ].…”

Section: Introductionmentioning

confidence: 99%

Real-Time 3D Reconstruction Method Based on Monocular Vision

Jia

Chang

Qiang

et al. 2021

Sensors

View full text Add to dashboard Cite

Real-time 3D reconstruction is one of the current popular research directions of computer vision, and it has become the core technology in the fields of virtual reality, industrialized automatic systems, and mobile robot path planning. Currently, there are three main problems in the real-time 3D reconstruction field. Firstly, it is expensive. It requires more varied sensors, so it is less convenient. Secondly, the reconstruction speed is slow, and the 3D model cannot be established accurately in real time. Thirdly, the reconstruction error is large, which cannot meet the requirements of scenes with accuracy. For this reason, we propose a real-time 3D reconstruction method based on monocular vision in this paper. Firstly, a single RGB-D camera is used to collect visual information in real time, and the YOLACT++ network is used to identify and segment the visual information to extract part of the important visual information. Secondly, we combine the three stages of depth recovery, depth optimization, and deep fusion to propose a three-dimensional position estimation method based on deep learning for joint coding of visual information. It can reduce the depth error caused by the depth measurement process, and the accurate 3D point values of the segmented image can be obtained directly. Finally, we propose a method based on the limited outlier adjustment of the cluster center distance to optimize the three-dimensional point values obtained above. It improves the real-time reconstruction accuracy and obtains the three-dimensional model of the object in real time. Experimental results show that this method only needs a single RGB-D camera, which is not only low cost and convenient to use, but also significantly improves the speed and accuracy of 3D reconstruction.

show abstract

Semantic Stereo Matching With Pyramid Cost Volumes

Cited by 95 publications

References 28 publications

Bidirectional Stereo Matching Network With Double Cost Volumes

Bidirectional Stereo Matching Network With Double Cost Volumes

A Fast Stereo Matching Network with Multi-Cross Attention

Real-Time 3D Reconstruction Method Based on Monocular Vision

Contact Info

Product

Resources

About