Single- and Multi-FPGA Acceleration of Dense Stereo Vision for Planetary Rovers

Abstract

Increased mobile autonomy is a vital requisite for future planetary exploration rovers. Stereo vision is a key enabling technology in this regard, as it can passively reconstruct in three dimensions the surroundings of a rover and facilitate the selection of science targets and the planning of safe routes. Nonetheless, accurate dense stereo algorithms are computationally demanding. When executed on the low-performance, radiation-hardened CPUs typically installed on rovers, slow stereo processing severely limits the driving speed and hence the science that can be conducted in situ . Aiming to decrease execution time while increasing the accuracy of stereo vision embedded in future rovers, this article proposes HW/SW co-design and acceleration on resource-constrained, space-grade FPGAs. In a top-down approach, we develop a stereo algorithm based on the space sweep paradigm, design its parallel HW architecture, implement it with VHDL, and demonstrate feasible solutions even on small-sized devices with our multi-FPGA partitioning methodology. To meet all cost, accuracy, and speed requirements set by the European Space Agency for this system, we customize our HW/SW co-processor by design space exploration and testing on a Mars-like dataset. Implemented on Xilinx Virtex technology, or European NG-MEDIUM devices, the FPGA kernel processes a 1,120 × 1,120 stereo pair in 1.7s−3.1s, utilizing only 5.4−9.3 LUT6 and 200−312 RAMB18. The proposed system exhibits up to 32× speedup over desktop CPUs, or 2,810× over space-grade LEON3, and achieves a mean reconstruction error less than 2cm up to 4m depth. Excluding errors exceeding 2cm (which are less than 4% of the total), the mean error is under 8mm.

Efficient autonomous navigation for planetary rovers with limited resources

“…VisLoc (Shaw, Woods, Churchill, & Newman, 2013), developed during the ExoMars mission and based on the Oxford visual odometry (OVO) algorithm (Churchill & Newman, 2012), and SPARTAN (Kostavelis, Boukas, Nalpantidis, Gasteratos, & AvilesRodrigalvarez, 2011, 2014), developed under a series of ESA R&D activities during the last decade. In addition, latest works in SPARTAN made a significant effort toward optimizing and efficiently bringing localization and mapping processes to space representative FPGA hardware implementations (Avilés et al, 2018; Lentaris, Maragos, Soudris, Zabulis, & Lourakis, 2019). ExoMars‐like field test campaigns have been run in the Atacama Desert to increase the technology readiness level (TRL) of these technologies.…”

Design methodologies, frameworks and implementations on reconfigurable devices for timing and energy optimizations

“…Specifically, we i) develop the first FPGA implementation of the plane sweep stereo vision algorithm in the literature, and ii) we introduce a multi-FPGA partitioning methodology for design mapping onto multiple FPGAs. We clarify that this work [77] is the product of European Space Agency (ESA) funded projects, namely SEXTANT 1 and COMPASS 2 , with the thesis author's contributions being on the HW implementation and HW analysis of specific components of plane sweep algorithm (in Fig. 3.2, MNCC calculator, depth map and temp store), the development of multi-FPGA partitioning methodology and the integration of HW architecture on the multi-FPGA platform.…”

“…3.2, MNCC calculator, depth map and temp store), the development of multi-FPGA partitioning methodology and the integration of HW architecture on the multi-FPGA platform. The SW development and analysis of the plane sweep algorithm was performed by Foundation for Research and Technology -Hellas (FORTH) and in-depth details can be found in [77].…”

“…For presentation purposes, we briefly describe the plane sweep algorithm which was developed by FORTH [77] as already mentioned in the introduction. The plane sweep stereo algorithm, hereafter denoted as psweep, uses a plane as its sweeping surface, all instances of which are parallel to each other.…”

“…For presentation purposes, we present the entire HW implementation and analysis of the psweep algorithm which is part of an ESA funded project [77] as described in the introduction. The HW implementation of psweep involves algorithmic profiling/analysis, HW/SW partitioning, HW architecture design, parametric VHDL coding, system integration with CPU-FPGA communication, as well as design space exploration, i.e.…”