SyntCities: A Large Synthetic Remote Sensing Dataset for Disparity Estimation

Abstract

Studies in the last years have proved the outstanding performance of deep learning for computer vision tasks in the remote sensing field, such as disparity estimation. However, available datasets mostly focus on close-range applications like autonomous driving or robot manipulation. To reduce the domain gap while training we present SyntCities, a synthetic dataset resembling the aerial imagery on urban areas. The pipeline used to render the images is based on 3D modelling, which helps to avoid acquisition costs, provides sub-pixel accurate dense ground truth and simulates different illumination conditions. The dataset additionally provides multi-class semantic maps and can be used converted to point cloud format to benefit a wider research community. We focus on the task of disparity estimation and evaluate the performance of the traditional Semi-Global Matching and state of the art architectures, trained with SyntCities and other datasets, on real aerial and satellite images. A comparison with the widely used SceneFlow dataset is also presented. Strategies using a mixture of both real and synthetic samples are studied as well. Results show significant improvements in terms of accuracy for the disparity maps. SyntCities can be downloaded at: https://tinyurl.com/77e3n6m9

Roof3d: A Real and Synthetic Data Collection for Individual Building Roof Plane and Building Sections Detection

“…Public datasets, such as those provided by the federal governments of North-Rhine-Westphalia (NRW), Germany 1 , include a large variety of annotated data, but the respective annotations have been detected automatically by a laser-scanning-based method, which leads to severe inaccuracies (see Figure 2). Using synthetic data, which is generated by a procedure as described in (Reyes et al, 2022) implies high-quality annotations of considerable quantity but leads to a domain gap (see Figure 2). Thus, mixing the synthetic data with real data is a potential approach to have both domain specificity and accurate labels.…”

“…Synthetic Data To both compensate the lack of labelled data and provide a highly accurate ground truth, we also created synthetic data resembling aerial imagery. We used a pipeline similar to the one in (Reyes et al, 2022) and a 3D model based on Paris from the ESRI platform 3 . We edited the model to include a larger density of buildings and a LoD-2 representation.…”

Ignorance is bliss: flawed assumptions in simulated ground truth

“…Generating ground truth is tedious, error-prone, and introduces human bias to training data making synthetic data from simulated environments a promising approach to alleviating these difficulties. Many recent synthetic ground truth data sets make similar promises in describing their ground truth: "pixel-perfect", 1 "pixelaccurate", 2 "sub pixel accurate", 3 "pixel-level". 4 This demonstrates a prevalence of thought where each pixel is treated as a little square.…”

An Evaluation of Stereo and Multiview Algorithms for 3d Reconstruction With Synthetic Data

“…We train the algorithms in the same datasets to set a fair comparison, for which the datasets have been properly adapted. We utilise the SyntCities dataset from our previous work (Fuentes Reyes et al, 2022), as this resembles remote sensing aerial imagery and provides all necessary input data for the selected algorithms and the Scene-Flow (Mayer et al, 2016) dataset, which has been widely used for training. Non-learning algorithms are considered as well for a comparable baseline.…”