IRS: A Large Naturalistic Indoor Robotics Stereo Dataset to Train Deep Models for Disparity and Surface Normal Estimation

Wang, Qiang; Zheng, Shizhen; Yan, Qingsong; Deng, Fei; Zhao, Kaixuan; Chu, Xiaowen

doi:10.48550/arxiv.1912.09678

Cited by 6 publications

(8 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We learn a monocular depth prediction network using a scale-and shift-invariant trimmed loss that operates on an inverse depth representation, together with the gradient-matching loss proposed in [22]. We construct a meta-dataset that includes the original datasets that were used in [30] (referred to as MIX 5 in that work) and extend it with with five additional datasets ( [18,43,44,46,47]). 1.…”

Section: Monocular Depth Estimationmentioning

confidence: 99%

Vision Transformers for Dense Prediction

Ranftl¹,

Bochkovskiy²,

Koltun³

2021

Preprint

View full text Add to dashboard Cite

We introduce dense vision transformers, an architecture that leverages vision transformers in place of convolutional networks as a backbone for dense prediction tasks. We assemble tokens from various stages of the vision transformer into image-like representations at various resolutions and progressively combine them into full-resolution predictions using a convolutional decoder. The transformer backbone processes representations at a constant and relatively high resolution and has a global receptive field at every stage. These properties allow the dense vision transformer to provide finer-grained and more globally coherent predictions when compared to fully-convolutional networks. Our experiments show that this architecture yields substantial improvements on dense prediction tasks, especially when a large amount of training data is available. For monocular depth estimation, we observe an improvement of up to 28% in relative performance when compared to a state-of-theart fully-convolutional network. When applied to semantic segmentation, dense vision transformers set a new state of the art on ADE20K with 49.02% mIoU. We further show that the architecture can be fine-tuned on smaller datasets such as NYUv2, KITTI, and Pascal Context where it also sets the new state of the art. Our models are available at https://github.com/intel-isl/DPT.

show abstract

Section: Monocular Depth Estimationmentioning

confidence: 99%

Vision Transformers for Dense Prediction

Ranftl¹,

Bochkovskiy²,

Koltun³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Simulation software such as UE and Blender are the most widely adopted tools, which allow researchers to build their own scenes, with changeable textures, lighting and weather conditions [26]. Representative datasets constructed using this method include Sintel [21], Scene Flow [20], HR-VS [27], IRS [14] and New Tsukuba CG [28]. In general, disparity images generated by simulation software have the advantages of high accuracy, high density (usually no invalid pixels), and convenient to build large scale datasets.…”

Section: A Stereo Matching Datasetmentioning

confidence: 99%

“…In terms of the accuracy of the disparity map, only three datasets: Middlebury2014 [7], HR-VS [27] and IRS [14] achieved sub-pixel accuracy, and the latter two [14], [27] were synthetic datasets generated by software. Obviously, as the average matching error of the SOTA deep learning stereo matching model has been less than one pixel, most datasets that only provide pixel-level accuracy disparity maps can no longer meet the requirements of the deep learning models.…”

Section: B Dataset Comparisonmentioning

confidence: 99%

“…We summarize the public stereo matching datasets in table III for comparison. ≈ 96 % KITTI [1], [2] LiDAR autonomous driving 389+400 pixel ≈ 19 % Scene Flow [20] software animation 39049 pixel × 100 % HR-VS [27] software autonomous driving 780 sub-pixel × 100 % New Tsukuba [28] software indoor 1800 pixel × 100 % ETH3D [12] scanner indoor/outdoor 27(train)+20(test) pixel ≈ 69 % DrivingStereo [3] LiDAR autonomous driving 174437(train)+7751(test) pixel ≈ 4 % InStereo2K [13] structured light indoor 2010(train)+50(test) pixel ≈ 87 % ApolloScape [4] LiDAR autonomous driving 4156(train)+1009(test) pixel ≈ 90 % Argoverse [5] LiDAR autonomous driving 4008(train)+1522(validate)+1094(test) pixel ≈ 0.86 % Sintel [21] software animation 1064 pixel × 100% IRS [14] software indoor 84946(train)+15079(test) sub-pixel × 100% CATS [15] LiDAR indoor/outdoor 1372 pixel ≈ 8% Ladicky [22] annotation autonomous driving 50(train)+20(test) pixel Cityscapes [6] SGM algorithm autonomous driving pixel ≈ 38%…”

Section: B Dataset Comparisonmentioning

confidence: 99%

See 1 more Smart Citation

PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction

Wang¹,

Ma²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Stereo matching is an important task in computer vision which has drawn tremendous research attention for decades. While in terms of disparity accuracy, density and data size, public stereo datasets are difficult to meet the requirements of models. In this paper, we aim to address the issue between datasets and models and propose a large scale stereo dataset with high accuracy disparity ground truth named PlantStereo. We used a semi-automatic way to construct the dataset: after camera calibration and image registration, high accuracy disparity images can be obtained from the depth images. In total, PlantStereo contains 812 image pairs covering a diverse set of plants: spinach, tomato, pepper and pumpkin. We firstly evaluated our PlantStereo dataset on four different stereo matching methods. Extensive experiments on different models and plants show that compared with ground truth in integer accuracy, high accuracy disparity images provided by PlantStereo can remarkably improve the training effect of deep learning models. This paper provided a feasible and reliable method to realize plant surface dense reconstruction. The PlantStereo dataset and relative code are available at: https://www.github.com/wangqingyu985/PlantStereo

show abstract

“…Additionally, we conduct another interesting evaluation. We use IRS dataset [29], a large synthetic stereo dataset, and Flyingthings3D together to train BGNet and BGNet+. IRS contains more than 100,000 pairs of 960 × 540 resolution stereo images (84,946 training and 15,079 testing) in indoor scenes.…”

Section: Generalization Performancementioning

confidence: 99%

Bilateral Grid Learning for Stereo Matching Networks

Xu¹,

Xu²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

The real-time performance of the stereo matching network is important for many applications, such as automatic driving, robot navigation and augmented reality (AR). Although significant progress has been made in stereo matching networks in recent years, it is still challenging to balance real-time performance and accuracy. In this paper, we present a novel edge-preserving cost volume upsampling module based on the slicing operation in the learned bilateral grid. The slicing layer is parameter-free, which allows us to obtain a high quality cost volume of high resolution from a low resolution cost volume under the guide of the learned guidance map efficiently. The proposed cost volume upsampling module can be seamlessly embedded into many existing stereo matching networks, such as GCNet, PSMNet, and GANet. The resulting networks are accelerated several times while maintaining comparable accuracy. Furthermore, based on this module we design a real-time network (named BGNet), which outperforms the existing published real-time deep stereo matching networks, as well as some complex networks on KITTI stereo datasets. The code of the proposed method will be available.

show abstract

IRS: A Large Naturalistic Indoor Robotics Stereo Dataset to Train Deep Models for Disparity and Surface Normal Estimation

Cited by 6 publications

References 40 publications

Vision Transformers for Dense Prediction

Vision Transformers for Dense Prediction

PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction

Bilateral Grid Learning for Stereo Matching Networks

Contact Info

Product

Resources

About