Real-Time Semantic Stereo Matching

Dovesi, Pier Luigi; Poggi, Matteo; Andraghetti, Lorenzo; Martí, Miquel; Pieropan, Alessandro; Mattoccia, Stefano

doi:10.1109/icra40945.2020.9196784

Cited by 56 publications

(17 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [19], a GPU architecture for real‐time semantic stereo matching was proposed. The proposed framework relied on coarse‐to‐fine estimations in a multi‐stage fashion, allowing: 1) Very fast inference even on embedded devices, with marginal drops in accuracy.…”

Section: Related Workmentioning

confidence: 99%

Real‐time multi‐window stereo matching algorithm with fuzzy logic

Vázquez-Delgado

Pérez-Patricio

Aguilar-González

et al. 2021

IET Computer Vision

View full text Add to dashboard Cite

Stereo matching obtains a depth map called a disparity map that indicates or shows the positions of the objects in a scene. To estimate a disparity map, the most popular trend consists of comparing two images (left‐right) from two different points from the same scene. Unfortunately, small window sizes are suitable to preserve the edges, while large window sizes are required in homogeneous areas. To solve this problem, in this article, a novel real‐time stereo matching algorithm embedded in an FPGA is proposed. The approach consists of estimating disparity maps with different window sizes by using the sum of absolute differences (SAD) as a local correlation metric. Once the disparity maps are obtained, the left‐right consistency for each window size is computed. At the end of this stage, the centre pixel deviation is estimated through a 5 × 5 window and the Sobel gradient is extracted from the left image. Finally, both parameters are processed by a Fuzzy Inference System (FIS), which combines the calculated disparities and generates a final disparity map. An architecture embedded in FPGA is established and hardware acceleration strategies are discussed. Experimental results demonstrated that this algorithmic formulation provides promising results compared with the current state of the art.

show abstract

Section: Related Workmentioning

confidence: 99%

Real‐time multi‐window stereo matching algorithm with fuzzy logic

Vázquez-Delgado

Pérez-Patricio

Aguilar-González

et al. 2021

IET Computer Vision

View full text Add to dashboard Cite

show abstract

“…In the first category, [22,32,15,53,54,46,1] extend the seminal DispNet [24], an end-to-end network for disparity regression. The second class, instead, consists of architectures that explicitly construct 3D feature cost volumes by means of concatenation/feature difference [4,18,47,48,8,57,49,55,29,3,7,45,50,21] and groupwise correlation [14]. A thorough review of these works can be found in [36].…”

Section: Related Workmentioning

confidence: 99%

SMD-Nets: Stereo Mixture Density Networks

Tosi¹,

Liao²,

Schmitt³

et al. 2021

Preprint

View full text Add to dashboard Cite

Despite stereo matching accuracy has greatly improved by deep learning in the last few years, recovering sharp boundaries and high-resolution outputs efficiently remains challenging. In this paper, we propose Stereo Mixture Density Networks (SMD-Nets), a simple yet effective learning framework compatible with a wide class of 2D and 3D architectures which ameliorates both issues. Specifically, we exploit bimodal mixture densities as output representation and show that this allows for sharp and precise disparity estimates near discontinuities while explicitly modeling the aleatoric uncertainty inherent in the observations. Moreover, we formulate disparity estimation as a continuous problem in the image domain, allowing our model to query disparities at arbitrary spatial precision. We carry out comprehensive experiments on a new high-resolution and highly realistic synthetic stereo dataset, consisting of stereo pairs at 8Mpx resolution, as well as on real-world stereo datasets. Our experiments demonstrate increased depth accuracy near object boundaries and prediction of ultra high-resolution disparity maps on standard GPUs. We demonstrate the flexibility of our technique by improving the performance of a variety of stereo backbones.

show abstract

“…This process is similar to the residual prediction and recurrence. Many existing works train models to predict residuals to refine an initial prediction [12], [13], [14], [15], [16]. However, these models update an initial prediction by a fixed number of steps or dimension scales.…”

Section: Related Workmentioning

confidence: 99%

ORStereo: Occlusion-Aware Recurrent Stereo Matching for 4K-Resolution Images

Hu¹,

Wang²,

Hou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Stereo reconstruction models trained on small images do not generalize well to high-resolution data. Training a model on high-resolution image size faces difficulties of data availability and is often infeasible due to limited computing resources. In this work, we present the Occlusion-aware Recurrent binocular Stereo matching (ORStereo), which deals with these issues by only training on available low disparity range stereo images. ORStereo generalizes to unseen highresolution images with large disparity ranges by formulating the task as residual updates and refinements of an initial prediction. ORStereo is trained on images with disparity ranges limited to 256 pixels, yet it can operate 4K-resolution input with over 1000 disparities using limited GPU memory. We test the model's capability on both synthetic and real-world high-resolution images. Experimental results demonstrate that ORStereo achieves comparable performance on 4K-resolution images compared to state-of-the-art methods trained on large disparity ranges. Compared to other methods that are only trained on low-resolution images, our method is 70% more accurate on 4K-resolution images.

show abstract

Real-Time Semantic Stereo Matching

Cited by 56 publications

References 69 publications

Real‐time multi‐window stereo matching algorithm with fuzzy logic

Real‐time multi‐window stereo matching algorithm with fuzzy logic

SMD-Nets: Stereo Mixture Density Networks

ORStereo: Occlusion-Aware Recurrent Stereo Matching for 4K-Resolution Images

Contact Info

Product

Resources

About