TartanAir: A Dataset to Push the Limits of Visual SLAM

Wang, Wenshan; Zhu, Delong; Wang, Xiangwei; Hu, Yaoyu; Qiu, Yuheng; Wang, Chen; Hu, Yafei; Kapoor, Ashish; Scherer, Sebastian

doi:10.48550/arxiv.2003.14338

Cited by 15 publications

(10 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This indicate a significant variance on the performance of [17]. Actually, this is the limitation of most deep learning based methods, which has been discussed in detail by [18].…”

Section: B Quantitative Evaluationmentioning

confidence: 98%

“…In recent work [14]- [18], deep learning based MVO algorithms are proposed, in which the camera pose with a real scale is directly predicted by the neural network in an end-to-end manner. Such methods have received much attention in recent years, but their generalization ability across different scenarios is very limited [18]. Some other deep learning based methods take scale recovery as an independent problem.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Accurate and Robust Scale Recovery for Monocular Visual Odometry Based on Plane Geometry

Tian

Zhang

Zhu

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Scale ambiguity is a fundamental problem in monocular visual odometry. Typical solutions include loop closure detection and environment information mining. For applications like self-driving cars, loop closure is not always available, hence mining prior knowledge from the environment becomes a more promising approach. In this paper, with the assumption of a constant height of the camera above the ground, we develop a light-weight scale recovery framework leveraging an accurate and robust estimation of the ground plane. The framework includes a ground point extraction algorithm for selecting high-quality points on the ground plane, and a ground point aggregation algorithm for joining the extracted ground points in a local sliding window. Based on the aggregated data, the scale is finally recovered by solving a least-squares problem using a RANSAC-based optimizer. Sufficient data and robust optimizer enable a highly accurate scale recovery. Experiments on the KITTI dataset show that the proposed framework can achieve state-of-the-art accuracy in terms of translation errors, while maintaining competitive performance on the rotation error. Due to the light-weight design, our framework also demonstrates a high frequency of 20Hz on the dataset.

show abstract

“…This indicate a significant variance on the performance of [17]. Actually, this is the limitation of most deep learning based methods, which has been discussed in detail by [18].…”

Section: B Quantitative Evaluationmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

Accurate and Robust Scale Recovery for Monocular Visual Odometry Based on Plane Geometry

Tian

Zhang

Zhu

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

show abstract

“…ORStereo allows us to train with only small-sized images and a typical disparity range around 200 pixels. We utilize several public datasets for the MCUA [28] Bi3D [29] GwcNet [30] FADNet [31] GA-Net [32] WaveletStereo [33] DeepPruner [34] SSPCV-Net [35] AANet [ training, i.e., the Middlebury dataset at 1/4 resolution [2], the Scene Flow [5] dataset (∼35k stereo pairs), and the TartanAir [36] datasets (∼18k pairs sampled). The Scene Flow and TartanAir datasets do not provide true occlusion labels.…”

Section: A Datasets and Details Of Trainingmentioning

confidence: 99%

ORStereo: Occlusion-Aware Recurrent Stereo Matching for 4K-Resolution Images

Hu¹,

Wang²,

Hou³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Stereo reconstruction models trained on small images do not generalize well to high-resolution data. Training a model on high-resolution image size faces difficulties of data availability and is often infeasible due to limited computing resources. In this work, we present the Occlusion-aware Recurrent binocular Stereo matching (ORStereo), which deals with these issues by only training on available low disparity range stereo images. ORStereo generalizes to unseen highresolution images with large disparity ranges by formulating the task as residual updates and refinements of an initial prediction. ORStereo is trained on images with disparity ranges limited to 256 pixels, yet it can operate 4K-resolution input with over 1000 disparities using limited GPU memory. We test the model's capability on both synthetic and real-world high-resolution images. Experimental results demonstrate that ORStereo achieves comparable performance on 4K-resolution images compared to state-of-the-art methods trained on large disparity ranges. Compared to other methods that are only trained on low-resolution images, our method is 70% more accurate on 4K-resolution images.

show abstract

“…In this section, we evaluate our proposed method using two publicly available datasets for robotics: the KITTI dataset [73] and the TartanAir dataset [78]. For both datasets, qualitative and quantitative results are provided with discussion.…”

Section: Experimental Evaluationmentioning

confidence: 99%

“…As KITTI data set aims for on-road autonomous vehicle applications, the environments are semblable and relatively easy. To further test our framework in other environments, we employ a new challenging dataset, the TartanAir [78]. TartanAir is mainly introduced as a benchmark for visual SLAM algorithms, but it is also suitable for other robotic applications such as robotic mapping.…”

Section: Tartanair Datasetmentioning

confidence: 99%

Multi-Task Learning for Scalable and Dense Multi-Layer Bayesian Map Inference

Gan¹,

Kim²,

Grizzle³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper presents a novel and flexible multi-task multi-layer Bayesian mapping framework with readily extendable attribute layers. The proposed framework goes beyond modern metric-semantic maps to provide even richer environmental information for robots in a single mapping formalism while exploiting existing inter-layer correlations. It removes the need for a robot to access and process information from many separate maps when performing a complex task and benefits from the correlation between map layers, advancing the way robots interact with their environments. To this end, we design a multi-task deep neural network with attention mechanisms as our front-end to provide multiple observations for multiple map layers simultaneously. Our back-end runs a scalable closedform Bayesian inference with only logarithmic time complexity. We apply the framework to build a dense robotic map including metric-semantic occupancy and traversability layers. Traversability ground truth labels are automatically generated from exteroceptive sensory data in a self-supervised manner. We present extensive experimental results on publicly available data sets and data collected by a 3D bipedal robot platform on the University of Michigan North Campus and show reliable mapping performance in different environments. Finally, we also discuss how the current framework can be extended to incorporate more information such as friction, signal strength, temperature, and physical quantity concentration using Gaussian map layers. The software for reproducing the presented results or running on customized data is made publicly available.

show abstract

TartanAir: A Dataset to Push the Limits of Visual SLAM

Cited by 15 publications

References 31 publications

Accurate and Robust Scale Recovery for Monocular Visual Odometry Based on Plane Geometry

Accurate and Robust Scale Recovery for Monocular Visual Odometry Based on Plane Geometry

ORStereo: Occlusion-Aware Recurrent Stereo Matching for 4K-Resolution Images

Multi-Task Learning for Scalable and Dense Multi-Layer Bayesian Map Inference

Contact Info

Product

Resources

About