Next-Best Stereo: Extending Next-Best View Optimisation For Collaborative Sensors

Abstract

Most 3D reconstruction approaches passively optimise over all data, exhaustively matching pairs, rather than actively selecting data to process. This is costly both in terms of time and computer resources, and quickly becomes intractable for large datasets.This work proposes an approach to intelligently filter large amounts of data for 3D reconstructions of unknown scenes using monocular cameras. Our contributions are twofold: First, we present a novel approach to efficiently optimise the Next-Best View (NBV) in terms of accuracy and coverage using partial scene geometry. Second, we extend this to intelligently selecting stereo pairs by jointly optimising the baseline and vergence to find the NBV's best stereo pair to perform reconstruction. Both contributions are extremely efficient, taking 0.8ms and 0.3ms per pose, respectively.Experimental evaluation shows that the proposed method allows efficient selection of stereo pairs for reconstruction, such that a dense model can be obtained with only a small number of images. Once a complete model has been obtained, the remaining computational budget is used to intelligently refine areas of uncertainty, achieving results comparable to state-of-the-art batch approaches on the Middlebury dataset, using as little as 3.8% of the views.

PredRecon: A Prediction-boosted Planning Framework for Fast and High-quality Autonomous Aerial Reconstruction

“…Many previous studies [9,30,31] demonstrated the highquality MVS reconstruction thoroughly depending on the following factors, including visibility S vis , relative distance S dis and triangulation angle S ang , presented in Eq.10, 11, 12, 13. To optimize MVS performance of a local path, we decompose the MVS structure into several basic triangulation units, which is defined as each of two neighboring viewpoints in the local path with their co-visible cluster surface.…”

3D-NVS: A 3D Supervision Approach for Next View Selection

“…Daudelin et al [9] propose a probabilistic NVS algorithm but use RGB-D sensor data for 3D object reconstruction. Mendez et al [25] use stereo image pairs and propose NVS to optimize the surface area covered by reconstructing the object surface. Dunn et al [12] use passive cameras but optimize for the surface area covered as their goal is surface reconstruction.…”