Edge supervision and multi-scale cost volume for stereo matching

Yang, Xiaowei; Feng, Zhiguo; Zhao, Yong; Zhang, Guiying; He, Lin

doi:10.1016/j.imavis.2021.104336

Cited by 14 publications

(10 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To evaluate the performance of our method, we compare LWNet on the KITTI 2015 testing dataset with other state‐of‐the‐art networks (i.e. high computational cost and high accuracy), including LEAStereo [39], ACVNet [40], RDNet [8], CFNet [41], GwcNet‐gc [9], PSMNet [7], MANet [42], GC‐Net [20], GANet‐15 [43], AcfNet [44] and AANet [45]. We integrate all these methods and measure the inference time with a 1242 × 375 resolution on a single NVIDIA 3090 GPU.…”

Section: Methodsmentioning

confidence: 99%

“…Yang et al. [8] and Song et al. [21] proposed a multi‐task network which incorporates the edge clue into stereo matching.…”

Section: Related Workmentioning

confidence: 99%

“…Over the past few years, deep convolutional neural networks (CNNs) have been able to extract more complex image features and have been applied to learn how to match corresponding points [6]. Although current CNN-based stereo matching methods yield significant performance gains compared to conventional algorithms in terms of both accuracy and speed, most such methods [7][8][9] contain a deep number of layers and exploit a 3D CNN for regularizing cost volume to achieve an accurate disparity regression calculation, which results in too many parameters, and they are not suited to apply on low-budget devices. Some networks [10,11] compress the channel dimension of the entire learning feature and only use 2D convolution for cost aggregation, which lead to a loss of feature information and reduce the accuracy of stereo matching.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A light‐weight stereo matching network based on multi‐scale features fusion and robust disparity refinement

Yang

Zhao

Feng

et al. 2023

IET Image Processing

Self Cite

View full text Add to dashboard Cite

In recent years, convolutional‐neural‐network based stereo matching methods have achieved significant gains compared to conventional methods in terms of both speed and accuracy. Current state‐of‐the‐art disparity estimation algorithms require many parameters and large amounts of computational resources and are not suited for applications on edge devices. In this paper, an end‐to‐end light‐weight network (LWNet) for fast stereo matching is proposed, which consists of an efficient backbone with multi‐scale feature fusion for feature extraction, a 3D U‐Net aggregation architecture for disparity computation, and color guidance in a 2D convolutional neural network (CNN) for disparity refinement. MobileNetV2 is adopted as an efficient backbone in feature extraction. The channel attention module is applied to improve the representational capacity of features and multi‐resolution information is adaptively incorporated into the cost volume via cross‐scale connections. Further, a left‐right consistency check and color guidance refinement are introduced and a robust disparity refinement network is designed with skip connections and dilated convolutions to capture global context information and improve disparity estimation accuracy with little computational cost and memory space. Extensive experiments on Scene Flow, KITTI 2015, and KITTI 2012 demonstrate that the proposed LWNet achieves competitive accuracy and speed when compared with state‐of‐the‐art stereo matching methods.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Yang et al. [8] and Song et al. [21] proposed a multi‐task network which incorporates the edge clue into stereo matching.…”

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A light‐weight stereo matching network based on multi‐scale features fusion and robust disparity refinement

Yang

Zhao

Feng

et al. 2023

IET Image Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…For instance, point cloud [ 6 ] characteristics are added to object identification models in autonomous driving to provide a multi-modal framework that makes up for the absence of depth in images. Multiple tasks sharing comparable cues would improve overall performance in a heuristic manner, particularly in edges, according to certain research that combines image disparity estimation and semantic segmentation together [ 7 ]. It is obvious that datasets with structural information are crucial because all of the methods mentioned above attempt to employ additional information to generate implicit depth cues.…”

Section: Introductionmentioning

confidence: 99%

A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation

Shen

Liu

Tian

et al. 2022

Sensors

View full text Add to dashboard Cite

Computer vision tasks, such as motion estimation, depth estimation, object detection, etc., are better suited to light field images with more structural information than traditional 2D monocular images. However, since costly data acquisition instruments are difficult to calibrate, it is always hard to obtain real-world scene light field images. The majority of the datasets for static light field images now available are modest in size and cannot be used in methods such as transformer to fully leverage local and global correlations. Additionally, studies on dynamic situations, such as object tracking and motion estimates based on 4D light field images, have been rare, and we anticipate a superior performance. In this paper, we firstly propose a new static light field dataset that contains up to 50 scenes and takes 8 to 10 perspectives for each scene, with the ground truth including disparities, depths, surface normals, segmentations, and object poses. This dataset is larger scaled compared to current mainstream datasets for depth estimation refinement, and we focus on indoor and some outdoor scenarios. Second, to generate additional optical flow ground truth that indicates 3D motion of objects in addition to the ground truth obtained in static scenes in order to calculate more precise pixel level motion estimation, we released a light field scene flow dataset with dense 3D motion ground truth of pixels, and each scene has 150 frames. Thirdly, by utilizing the DistgDisp and DistgASR, which decouple the angular and spatial domain of the light field, we perform disparity estimation and angular super-resolution to evaluate the performance of our light field dataset. The performance and potential of our dataset in disparity estimation and angular super-resolution have been demonstrated by experimental results.

show abstract

“…Cracks occupy much fewer pixels compared to the background area in the crack image. In this article, we propose a new crack detection algorithm based on boundary information 12 that can achieve a balance between speed and accuracy. In order to respond to this problem, this paper decides to post‐process the feature map which the STDC‐Net 11 outputs to reduce the error of detection.…”

Section: Introductionmentioning

confidence: 99%

Automatic bridge crack detection using boundary refinement based on real‐time segmentation network

Liu

Fang

et al. 2022

Structural Contr & Hlth

View full text Add to dashboard Cite

Summary The appearance of surface cracks is an important feature of the early safety of the architecture being threatened, which is of great significance to maintaining the safety of the bridge. Bridge safety maintenance is still a challenging issue owing to noise interference and unclear bridge images. The detection method based on deep learning has achieved remarkable result in semantic segmentation, target detection, and other fields, but still be unable to focus on balancing speed and accuracy. Wherefore, we designed a new crack detection network‐dense boundary refinement network (DBR‐Net), which combines the advantages of STDC‐Net (Short‐Term Dense Concatenate network) and refinement network. STDC‐Net mainly improves the detection rate by eliminating redundant structures and optimizes the detailed information by binary cross‐entropy loss and dice loss. Finally, the detail aggregation module combines the shallow spatial details with the deep semantic information to predict the segmentation results. Among them, the refinement network refines the segmentation results of the offset map generated by the training stage and the test stage. We verified the feasibility of DBR‐Net on three datasets. At the same time, the detection accuracy rate on our self‐made crack dataset reached 97.54%. What's more worth mentioning is that our detection rate has reached 37.0 images per second (IPS), realizing real‐time crack detection.

show abstract

Edge supervision and multi-scale cost volume for stereo matching

Cited by 14 publications

References 3 publications

A light‐weight stereo matching network based on multi‐scale features fusion and robust disparity refinement

A light‐weight stereo matching network based on multi‐scale features fusion and robust disparity refinement

A New Parallel Intelligence Based Light Field Dataset for Depth Refinement and Scene Flow Estimation

Automatic bridge crack detection using boundary refinement based on real‐time segmentation network

Contact Info

Product

Resources

About