Accurate omnidirectional multi-camera embedded structure from motion

Abstract

Trajectory estimation and 3D scene reconstruction from multiple cameras (also referred as Structure from Motion, SfM) will have a central role in the future of automotive industry. Typical appliance fields will be: autonomous navigation/guidance, collisions avoidance against static or moving objects (in particular pedestrians), parking assisted maneuvers and many more. The work exposed in this paper had mainly two different goals: (1) to describe the implementation of a real time embedded SfM modular pipeline featuring a dedicated optimized HW/SW system partitioning. It included also nonlinear optimizations such as local and global bundle adjustment at different stages of the pipeline; (2) to demonstrate quantitatively its performances on a synthetic test space specifically designed for its characterization. In order to make the system reliable and effective, providing the driver or the autonomous vehicle with a prompt response, the data rates and low latency of the 5G communication systems appear to make this choice the most promising communication solution

Modelling and automated calibration of a general multi‐projective camera

“…(); a stereo‐camera tracking algorithm was successfully tested by Gasparini and Bertolino (); multi‐camera‐based navigation for autonomous cars was discussed and demonstrated by Paracchini et al. (); and submarine visual tracking by a stereo camera was shown by Pfingsthorn et al. ().…”

“… 3D structure‐and‐motion estimation, for example, dense reconstruction (Furukawa and Ponce, ; Furukawa et al., ), sparse bundle adjustment and Internet‐based reconstruction (Snavely et al., ), space structure estimation (Se et al., ), incremental structure from motion (Wu, ), stereo panoramas (Amini et al., ) and dual fluoroscopy imaging system calibration used for bone reconstruction (Lichti et al., ). Navigation systems, for example, indoor localisation (Svoboda et al., ), stereo tracking for dental surgery (Wang et al., ), car trajectory estimation (Paracchini et al., ), underwater navigation (Pfingsthorn et al., ) and spacecraft precise docking (Tweddle, ). Combinatory applications with other sensors, for example, outdoor SLAM (Schleicher et al., ). …”

“…Recently published literature on MCSs highlights a wide range of real-world and futuristic applications. For example, robotic navigation by stereo vision was proposed by many researchers such as Grosso and Tistarelli (1993), Desouza and Kak (2002) or English et al (2014); a multi-camera-based indoor navigation system was successfully demonstrated by Svoboda et al (2002); Se et al (2007) proposed a stereo-camera-based 3D modelling of space structures; a multi-camera-based outdoor simultaneous localisation and mapping (SLAM) was demonstrated by Schleicher et al (2009); a stereo-camera tracking algorithm was successfully tested by Gasparini and Bertolino (2013); multi-camera-based navigation for autonomous cars was discussed and demonstrated by Paracchini et al (2016); and submarine visual tracking by a stereo camera was shown by Pfingsthorn et al (2016).…”