Unsupervised Estimation of Monocular Depth and VO in Dynamic Environments via Hybrid Masks

Sun, Qiyu; Tang, Yang; Zhang, Chongzhen; Zhao, Chaoqiang; Qian, Feng; Kurths, Jürgen

doi:10.1109/tnnls.2021.3100895

Cited by 21 publications

(7 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among all the training modes, the unsupervised methods based on monocular images are the most attractive because plenty of training data are available. However, unsupervised methods are not as reliable as supervised ones and are more vulnerable to the dynamic variations in environments [2]. Though all these deep learning based depth estimation methods dedicate to improve the performance of the model on a particular dataset, they neglect the transferability of the model, which imposes significant restrictions on real-world applications.…”

Section: A Depth Estimationmentioning

confidence: 99%

“…Monocular depth estimation is a classical task in computer vision, which aims to estimate the distances between the objects in environment and the agent itself [1], [2], and thus it is an essential task in environmental perception [3], [4]. Recently, deep learning-based depth estimation methods, including supervised methods [5] and unsupervised methods [1], [2], [6], are proposed and achieve significant progress. The supervised methods are trained through images with ground truth and their performances are often reliable.…”

Section: Introductionmentioning

confidence: 99%

“…Both real images collected by cameras and synthetic images generated by graphics engines [23], [24] are utilized for training in this work. In view of the fact that the unsupervised pipeline is vulnerable to the dynamics in environment [2], we propose a constraint to compel the depth maps estimated in different adversarial tasks to be identical. We name it as the cross-task depth consistency constraint and it is applied in meta-update.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learn to Adapt for Monocular Depth Estimation

Sun¹,

Yen²,

Yang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Monocular depth estimation is one of the fundamental tasks in environmental perception and has achieved tremendous progress in virtue of deep learning. However, the performance of trained models tends to degrade or deteriorate when employed on other new datasets due to the gap between different datasets. Though some methods utilize domain adaptation technologies to jointly train different domains and narrow the gap between them, the trained models cannot generalize to new domains that are not involved in training. To boost the transferability of depth estimation models, we propose an adversarial depth estimation task and train the model in the pipeline of meta-learning. Our proposed adversarial task mitigates the issue of meta-overfitting, since the network is trained in an adversarial manner and aims to extract domain invariant representations.In addition, we propose a constraint to impose upon cross-task depth consistency to compel the depth estimation to be identical in different adversarial tasks, which improves the performance of our method and smoothens the training process. Experiments demonstrate that our method adapts well to new datasets after few training steps during the test procedure.

show abstract

Section: A Depth Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learn to Adapt for Monocular Depth Estimation

Sun¹,

Yen²,

Yang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, supervised depth estimation methods need to collect a large amount of realdepth information data and require an immense amount of computing time in the training process, which greatly increases the difficulty and complexity of the algorithm. Comparatively speaking, unsupervised monocular depth estimation only requires monocular video sequences or stereo image pairs to realize the depth information estimation of each pixel of a single image [5][6][7]. In recent years, unsupervised monocular depth estimation have been favored by researchers [8][9][10].…”

Section: Introductionmentioning

confidence: 99%

Accurate unsupervised monocular depth estimation for ill-posed region

Sun¹,

Qin²,

Yuan³

et al. 2023

Front. Phys.

View full text Add to dashboard Cite

Unsupervised monocular depth estimation is challenging in ill-posed regions, such as weak texture scenes, projection occlusion, and redundant error of detail information, etc. In this paper, in order to tackle these problems, an improved unsupervised monocular depth estimation method for the ill-posed region is proposed through cascading training depth estimation network and pose estimation network by loss function. Firstly, for the depth estimation network, a feature extraction network using asymmetric convolution is designed instead of traditional convolution, which strengthens the extraction of the feature information and improves the accuracy of the weak texture scenes. Meanwhile, a feature extraction network integrating multi-scale receptive fields with the structure of different scale convolution and dilated convolution stack is designed to increase the underlying receptive field of the depth estimation network, which strengthens the fusion ability of the network for multi-scale detail information, and improves the integrity of the model output details. Secondly, a pose estimation network using an attention mechanism is presented to strengthen the pose detail information of keyframes and suppress redundant errors of the pose information of non-keyframes. Finally, a loss function with minimum reprojection error is adopted to alleviate the occlusion problem of the projection process between adjacent pixels and enhance the quality of the output depth images of the model. The experiments demonstrate that our method achieves state-of-the-art performance on KITTI monocular datasets.

show abstract

“…The unstable point features in the dynamic region were eliminated to ensure that only stable static point features were retained in the optimization part. Recently, learning-based computer visual methods are emerging, and also many feasible solutions [31] [32] to handle the impact of dynamic scenes on visual odometry by combining learning methods. However, the above-mentioned solutions all have strict requirements on scenario conditions and computer resources.…”

Section: Introductionmentioning

confidence: 99%

DynPL-SVO: A New Method Using Point and Line Features for Stereo Visual Odometry in Dynamic Scenes

Ma¹,

Wang²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Stereo visual odometry is widely used where a robot tracks its position and orientation using stereo cameras. Most of the approaches recovered mobile robotics motion based on the matching and tracking of point features along a sequence of stereo images. But in low-textured and dynamic scenes, there are no sufficient robust static point features for motion estimation, causing lots of previous work to fail to reconstruct the robotic motion. However, line features can be detected in such low-textured and dynamic scenes. In this paper, we proposed DynPL-SVO, a stereo visual odometry with the dynamic grid algorithm and the cost function containing both vertical and horizontal information of the line features. Stereo camera motion was obtained through Levenberg-Marquard minimization of reprojection error of point and line features. The experimental results on the KITTI and EuRoC MAV datasets showed that the DynPL-SVO had a competitive performance when compared to other state-of-the-art systems by producing more robust and accurate motion estimation, especially in low-textured and dynamic scenes.

show abstract

Unsupervised Estimation of Monocular Depth and VO in Dynamic Environments via Hybrid Masks

Cited by 21 publications

References 51 publications

Learn to Adapt for Monocular Depth Estimation

Learn to Adapt for Monocular Depth Estimation

Accurate unsupervised monocular depth estimation for ill-posed region

DynPL-SVO: A New Method Using Point and Line Features for Stereo Visual Odometry in Dynamic Scenes

Contact Info

Product

Resources

About