Efficient Multi-Scale Stereo-Matching Network Using Adaptive Cost Volume Filtering

Jeon, Sang‐Min; Heo, Yong Seok

doi:10.3390/s22155500

Cited by 5 publications

(5 citation statements)

References 35 publications

(123 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreovers, the regression or classification by constructing the single scale cost may lead to redundant or insufficient feature information, the model may be overfitting on a certain domain, and the robustness of the algorithm may be affected. As explained in related works 46 – 48 , multi-scale feature information can be utilized to obtain multiple receptive fields. Jeon et al 46 proposed an efficient multi-scale sequential feature fusion network to fully regularize the cost volume.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…As explained in related works 46 – 48 , multi-scale feature information can be utilized to obtain multiple receptive fields. Jeon et al 46 proposed an efficient multi-scale sequential feature fusion network to fully regularize the cost volume. MSCVNet 47 first generates multiple 3D cost volumes with different resolutions for cost aggregation.…”

Section: Proposed Methodsmentioning

confidence: 99%

See 1 more Smart Citation

An application of stereo matching algorithm based on transfer learning on robots in multiple scenes

Bi,

Li,

Tong

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Robot vision technology based on binocular vision holds tremendous potential for development in various fields, including 3D scene reconstruction, target detection, and autonomous driving. However, current binocular vision methods used in robotics engineering have limitations such as high costs, complex algorithms, and low reliability of the generated disparity map in different scenes. To overcome these challenges, a cross-domain stereo matching algorithm for binocular vision based on transfer learning was proposed in this paper, named Cross-Domain Adaptation and Transfer Learning Network (Ct-Net), which has shown valuable results in multiple robot scenes. First, this paper introduces a General Feature Extractor to extract rich general feature information for domain adaptive stereo matching tasks. Then, a feature adapter is used to adapt the general features to the stereo matching network. Furthermore, a Domain Adaptive Cost Optimization Module is designed to optimize the matching cost. A disparity score prediction module was also embedded to adaptively adjust the search range of disparity and optimize the cost distribution. The overall framework was trained using a phased strategy, and ablation experiments were conducted to verify the effectiveness of the training strategy. Compared with the prototype PSMNet, on KITTI 2015 benchmark, the 3PE-fg of Ct-Net in all regions and non-occluded regions decreased by 19.3 and 21.1% respectively, meanwhile, on the Middlebury dataset, the proposed algorithm improves the sample error rate at least 28.4%, which is the Staircase sample. The quantitative and qualitative results obtained from Middlebury, Apollo, and other datasets demonstrate that Ct-Net significantly improves the cross-domain performance of stereo matching. Stereo matching experiments in real-world scenes have shown that it can effectively address visual tasks in multiple scenes.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Section: Proposed Methodsmentioning

confidence: 99%

An application of stereo matching algorithm based on transfer learning on robots in multiple scenes

Bi,

Li,

Tong

et al. 2023

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Moreovers, the regression or classification by constructing the single scale cost may lead to redundant or insufficient feature information, the model may be overfitting on a certain domain, and the robustness of the algorithm may be affected. As explained in related works [46][47][48] , multi-scale feature information can be utilized to obtain multiple receptive fields. Jeon et al 46 proposed an efficient multi-scale sequential feature fusion network to fully regularize the cost volume.…”

Section: Domain Adaptive Cost Optimizationmentioning

confidence: 99%

“…As explained in related works [46][47][48] , multi-scale feature information can be utilized to obtain multiple receptive fields. Jeon et al 46 proposed an efficient multi-scale sequential feature fusion network to fully regularize the cost volume. MSCVNet 47 first generates multiple 3D cost volumes with different resolutions for cost aggregation.…”

Section: Domain Adaptive Cost Optimizationmentioning

confidence: 99%

An application of stereo matching algorithm based on transfer learning on robots in multiple scenarios

Wang

2023

Preprint

View full text Add to dashboard Cite

Robot vision technology based on binocular vision has significant potential for development in various fields, including 3D scenes and reconstruction, target detection, autonomous driving, and other fields. To date, current binocular vision methods used in robotics engineering suffer from high costs, complex algorithms, and low reliability of the generated disparity map in multiple scenarios. Robots require a cost-effective algorithm with cross-domain generalization capabilities for multiple scenarios. To address these issues, a cross-domain stereo matching algorithm for binocular vision based on transfer learning was proposed in this paper, named Cross-Domain Adaptation and Transfer Learning Network (Ct-Net), which has shown valuable results in multiple robot scenes. First, this paper introduces a General Feature Extractor (GFE) to extract rich general feature information for domain adaptive stereo matching tasks. Then, a feature adapter is used to adapt the general features to the stereo matching network. Furthermore, a Domain Adaptive Cost Optimization Module (DACOM) was designed to optimize the matching cost. A disparity score prediction module was also embedded to adaptively adjust the search range of disparity and optimize the cost distribution. The overall framework was trained using a phased strategy, and ablation experiments were conducted to verify the effectiveness of the training strategy. On KITTI 2015 benchmark, compared with the prototype PSMNet, the 3PE − fg of Ct-Net in all regions and non-occluded regions decreased by 19.3% and 21.1% respectively. On the Middlebury dataset, the 2PE of Ct-Net achieved comparable results on all samples. The quantitative and qualitative results obtained from Middlebury, Apollo, and other datasets demonstrate that Ct-Net significantly improves the cross-domain performance of stereo matching. Stereo matching experiments in real-world scenarios have shown that it can effectively address visual tasks in multiple robot scenes.

show abstract

“…The horizontal distance (

) between the corresponding pixel (or region) of the two images is defined as the disparity. The disparity is converted to depth using physical information from the capturing equipment, and the 3D position is then estimated from the depth [ 33 , 34 ].…”

Section: Related Workmentioning

confidence: 99%

Robust Estimation and Optimized Transmission of 3D Feature Points for Computer Vision on Mobile Communication Network

Kim

Park

Kim

et al. 2022

Sensors

View full text Add to dashboard Cite

Due to the amount of transmitted data and the security of personal or private information in wireless communication, there are cases where the information for a multimedia service should be directly transferred from the user’s device to the cloud server without the captured original images. This paper proposes a new method to generate 3D (dimensional) keypoints based on a user’s mobile device with a commercial RGB camera in a distributed computing environment such as a cloud server. The images are captured with a moving camera and 2D keypoints are extracted from them. After executing feature extraction between continuous frames, disparities are calculated between frames using the relationships between matched keypoints. The physical distance of the baseline is estimated by using the motion information of the camera, and the actual distance is calculated by using the calculated disparity and the estimated baseline. Finally, 3D keypoints are generated by adding the extracted 2D keypoints to the calculated distance. A keypoint-based scene change method is proposed as well. Due to the existing similarity between continuous frames captured from a camera, not all 3D keypoints are transferred and stored, only the new ones. Compared with the ground truth of the TUM dataset, the average error of the estimated 3D keypoints was measured as 5.98 mm, which shows that the proposed method has relatively good performance considering that it uses a commercial RGB camera on a mobile device. Furthermore, the transferred 3D keypoints were decreased to about 73.6%.

show abstract

Efficient Multi-Scale Stereo-Matching Network Using Adaptive Cost Volume Filtering

Cited by 5 publications

References 35 publications

An application of stereo matching algorithm based on transfer learning on robots in multiple scenes

An application of stereo matching algorithm based on transfer learning on robots in multiple scenes

An application of stereo matching algorithm based on transfer learning on robots in multiple scenarios

Robust Estimation and Optimized Transmission of 3D Feature Points for Computer Vision on Mobile Communication Network

Contact Info

Product

Resources

About