The Production of Ground Truths for Evaluating Highly Accurate Stereovision Algorithms

Abstract

The conception and improvement of algorithms for subpixel stereovision requires very precise test databases. The state of the art on the sets of images used extensively by the scientific community shows that they are often incomplete and imprecise compared to the dataset goals. We will present a method based on image synthesis to produce stereoscopic pairs with ground truths such as disparity and occlusion maps reaching an accuracy of about 10 −6 pixels. The a priori noise estimate is also taken into account. This process allows us to deliver a new image database consisting of 66 stereo pairs together with their ground truths.Keywords: ground truths; disparity map; stereovision; synthetic images Source CodeWe provide the code of a program that computes the 3D coordinates of the points associated with each ground truth in order to view them with appropriate software. We also provide a modified code of the algorithm from Lisani et al. [15]. It allows to process images in floating point format such as TIF or EXR. Supplementary MaterialThe database is available at this address 1 . The use of these images for scientists is permitted provided that this article is mentioned as well as the designers of the scenes.

Holopix50k: A Large-Scale In-the-wild Stereo Image Dataset

“…For example, the KITTI datasets were mainly developed for self-driving vehicle use-cases, while the few scenes in Middlebury are all captured in a laboratory setting. Other real-world stereo datasets include Make3D [47], ETH3D [49], CMLA [9], and Cityscapes [8] -each focused on a specific domain. More recent datasets such as Flickr1024 [57] and WSVD [55] provide more diverse scenes, however, Flickr1024 is relatively small compared to our dataset and WSVD images score significantly lower on quality metrics, as shown in Table 1.…”

“…A majority of state-of-the-art methods in this domain are based on deep learning methods, where accuracy and quality scale with the amount of data available for training. Current datasets either cover a limited subset of real-world scenarios [16] [8] or are taken in a laboratory setting [48] [9]. Further, there is an increased need for a large-scale stereo dataset representative of the diversity of real-world scenarios to enable generalization.…”

Comparison of Optical Flow Methods under Stereomatching with Short Baselines

“…The sequences used in this section belong to the CMLA dataset [5] and were produced by simulating a fronto-parallel camera motion. The images have near-zero noise and their disparity accuracy is in the order of 10 −6 .…”

“…On the one hand, this study implies the use of image databases with negligible noise levels and very accurate ground truths. Dagobert [5] found that commonly used databases poorly met these requirements and proposed a new one, the CMLA dataset. This database has the advantage of providing pairs of images created with different baselines, which have virtually no residual noise and dense disparity maps, whose accuracy is very high.…”