A light‐weight stereo matching network based on multi‐scale features fusion and robust disparity refinement

Abstract

In recent years, convolutional‐neural‐network based stereo matching methods have achieved significant gains compared to conventional methods in terms of both speed and accuracy. Current state‐of‐the‐art disparity estimation algorithms require many parameters and large amounts of computational resources and are not suited for applications on edge devices. In this paper, an end‐to‐end light‐weight network (LWNet) for fast stereo matching is proposed, which consists of an efficient backbone with multi‐scale feature fusion for feature extraction, a 3D U‐Net aggregation architecture for disparity computation, and color guidance in a 2D convolutional neural network (CNN) for disparity refinement. MobileNetV2 is adopted as an efficient backbone in feature extraction. The channel attention module is applied to improve the representational capacity of features and multi‐resolution information is adaptively incorporated into the cost volume via cross‐scale connections. Further, a left‐right consistency check and color guidance refinement are introduced and a robust disparity refinement network is designed with skip connections and dilated convolutions to capture global context information and improve disparity estimation accuracy with little computational cost and memory space. Extensive experiments on Scene Flow, KITTI 2015, and KITTI 2012 demonstrate that the proposed LWNet achieves competitive accuracy and speed when compared with state‐of‐the‐art stereo matching methods.

Adaptable 2D to 3D Stereo Vision Image Conversion Based on a Deep Convolutional Neural Network and Fast Inpaint Algorithm

“…These networks use well-known backbone architecture with U-net multi-level feature extractors [ 23 ]. Many methods use supervised training [ 24 , 25 , 26 , 27 , 28 ], while some use unsupervised neural network weight optimization [ 29 , 30 ]. Reviews on deep learning for monocular depth estimation can be found in references [ 31 , 32 , 33 , 34 , 35 , 36 ].…”