MobileStereoNet: Towards Lightweight Deep Networks for Stereo Matching

Shamsafar, Faranak; Woerz, Samuel; Rahim, Rafia; Zell, Andreas

doi:10.1109/wacv51458.2022.00075

Cited by 45 publications

(39 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…“Ours” means the EPE of the predicted disparity map of the proposed network. As shown in Table 2 , our MSFFNet outperforms all other 2D convolution-based models [ 10 , 17 , 27 , 28 ] and some 3D convolution-based models [ 12 , 13 ]. Among the 2D convolution networks, compared with SFFNet [ 17 ] which proposed the SFF module, the proposed MSFFNet shows better accuracy with

lower EPE.…”

Section: Experiments and Resultsmentioning

confidence: 83%

“…Figure 9 shows a qualitative comparison between the MSFFNet and other networks [ 12 , 17 , 26 , 28 ] on the Scene Flow test set. This comparison demonstrates that our network predicts disparity maps comparable to other 2D/3D convolution networks for most regions.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Figure 10 illustrates some of the qualitative comparison results of the MSFFNet and other networks [ 12 , 17 , 26 , 28 ] on the KITTI 2015 benchmark [ 25 ]. All figures are reported from the KITTI 2015 evaluation server.…”

Section: Experiments and Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Efficient Multi-Scale Stereo-Matching Network Using Adaptive Cost Volume Filtering

Jeon

Heo

2022

Sensors

View full text Add to dashboard Cite

While recent deep learning-based stereo-matching networks have shown outstanding advances, there are still some unsolved challenges. First, most state-of-the-art stereo models employ 3D convolutions for 4D cost volume aggregation, which limit the deployment of networks for resource-limited mobile environments owing to heavy consumption of computation and memory. Although there are some efficient networks, most of them still require a heavy computational cost to incorporate them to mobile computing devices in real-time. Second, most stereo networks indirectly supervise cost volumes through disparity regression loss by using the softargmax function. This causes problems in ambiguous regions, such as the boundaries of objects, because there are many possibilities for unreasonable cost distributions which result in overfitting problem. A few works deal with this problem by generating artificial cost distribution using only the ground truth disparity value that is insufficient to fully regularize the cost volume. To address these problems, we first propose an efficient multi-scale sequential feature fusion network (MSFFNet). Specifically, we connect multi-scale SFF modules in parallel with a cross-scale fusion function to generate a set of cost volumes with different scales. These cost volumes are then effectively combined using the proposed interlaced concatenation method. Second, we propose an adaptive cost-volume-filtering (ACVF) loss function that directly supervises our estimated cost volume. The proposed ACVF loss directly adds constraints to the cost volume using the probability distribution generated from the ground truth disparity map and that estimated from the teacher network which achieves higher accuracy. Results of several experiments using representative datasets for stereo matching show that our proposed method is more efficient than previous methods. Our network architecture consumes fewer parameters and generates reasonable disparity maps with faster speed compared with the existing state-of-the art stereo models. Concretely, our network achieves 1.01 EPE with runtime of 42 ms, 2.92M parameters, and 97.96G FLOPs on the Scene Flow test set. Compared with PSMNet, our method is 89% faster and 7% more accurate with 45% fewer parameters.

show abstract

lower EPE.…”

Section: Experiments and Resultsmentioning

confidence: 83%

Section: Experiments and Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Multi-Scale Stereo-Matching Network Using Adaptive Cost Volume Filtering

Jeon

Heo

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…YOLOStereo3D [26], a single-stage 3D detection network, effectively deals with the trade-off between computational complexity and depth accuracy with lightweight cost volume. Additionally, MobileStereoNet [38] introduces a way to leverage MobileNets [16] to reduce the computation cost of deep networks without sacrificing accuracy. From these observations, our method utilizes the disparity information as 3D geometric cues in multi-view settings to enhance the detection performance for the multi-view-based 3D object detection.…”

Section: Related Workmentioning

confidence: 99%

ORA3D: Overlap Region Aware Multi-view 3D Object Detection

Roh¹,

Chang²,

Moon³

et al. 2022

Preprint

View full text Add to dashboard Cite

In multi-view 3D object detection tasks, disparity supervision over overlapping image regions substantially improves the overall detection performance. However, current multi-view 3D object detection methods often fail to detect objects in the overlap region properly, and the network's understanding of the scene is often limited to that of a monocular detection network. To mitigate this issue, we advocate for applying the traditional stereo disparity estimation method to obtain reliable disparity information for the overlap region. Given the disparity estimates as a supervision, we propose to regularize the network to fully utilize the geometric potential of binocular images, and improve the overall detection accuracy. Moreover, we propose to use an adversarial overlap region discriminator, which is trained to minimize the representational gap between non-overlap regions and overlapping regions where objects are often largely occluded or suffer from deformation due to camera distortion, causing a domain shift. We demonstrate the effectiveness of the proposed method with the large-scale multi-view 3D object detection benchmark, called nuScenes. Our experiment shows that our proposed method outperforms the current state-of-the-art methods.

show abstract

“…Therefore, it can operate on depth maps Fig. 1: FusionLoc architecture diagram generated using stereo depth estimation techniques [12], [13], monocular depth estimation [14], [15], and infrared cameras such as the ones found in Kinect sensors.…”

Section: Introductionmentioning

confidence: 99%

Deep Camera Pose Regression Using Pseudo-LiDAR

Raza¹,

Lolic²,

Akhter³

et al. 2022

Preprint

View full text Add to dashboard Cite

An accurate and robust large-scale localization system is an integral component for active areas of research such as autonomous vehicles and augmented reality. To this end, many learning algorithms have been proposed that predict 6DOF camera pose from RGB or RGB-D images. However, previous methods that incorporate depth typically treat the data the same way as RGB images, often adding depth maps as additional channels to RGB images and passing them through convolutional neural networks (CNNs). In this paper, we show that converting depth maps into pseudo-LiDAR signals, previously shown to be useful for 3D object detection, is a better representation for camera localization tasks by projecting point clouds that can accurately determine 6DOF camera pose. This is demonstrated by first comparing localization accuracies of a network operating exclusively on pseudo-LiDAR representations, with networks operating exclusively on depth maps. We then propose FusionLoc, a novel architecture that uses pseudo-LiDAR to regress a 6DOF camera pose. FusionLoc is a dual stream neural network, which aims to remedy common issues with typical 2D CNNs operating on RGB-D images. The results from this architecture are compared against various other state-of-the-art deep pose regression implementations using the 7 Scenes dataset. The findings are that FusionLoc performs better than a number of other camera localization methods, with a notable improvement being, on average, 0.33m and 4.35 • more accurate than RGB-D PoseNet. By proving the validity of using pseudo-LiDAR signals over depth maps for localization, there are new considerations when implementing large-scale localization systems.

show abstract

MobileStereoNet: Towards Lightweight Deep Networks for Stereo Matching

Cited by 45 publications

References 26 publications

Efficient Multi-Scale Stereo-Matching Network Using Adaptive Cost Volume Filtering

Efficient Multi-Scale Stereo-Matching Network Using Adaptive Cost Volume Filtering

ORA3D: Overlap Region Aware Multi-view 3D Object Detection

Deep Camera Pose Regression Using Pseudo-LiDAR

Contact Info

Product

Resources

About