HW/SW Codesign and FPGA Acceleration of Visual Odometry Algorithms for Rover Navigation on Mars

Lentaris, George; Stamoulias, Ioannis; Soudris, Dimitrios; Lourakis, Manolis

doi:10.1109/tcsvt.2015.2452781

Cited by 43 publications

(25 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the fact that the scope of this review focuses on the ADR scenario, it is pointed out that other applications of vision-based navigation in space rely on analogous principles and employ similar visual primitives. Such applications, for example, include obstacle avoidance, mapping, and visual odometry for planetary rovers [17,18] or approach velocity estimation and localization for precision landing [18][19][20]. As a result, the choice of a processing platform suitable for ADR is also highly relevant to visual navigation applications for planetary exploration rovers and landers.…”

Section: Algorithms For Vision-based Navigation In Orbitmentioning

confidence: 99%

“…As a result, the choice of a processing platform suitable for ADR is also highly relevant to visual navigation applications for planetary exploration rovers and landers. Figure 2 depicts some indicative applications of VBN for rovers (top left) and ADR (bottom right); an image acquired by a Mars rover (top left) is processed by 3-D reconstruction (shown to its right, "mapping" via stereo matching) and visual odometry estimation (shown later, "localization" via feature detection and matching between successive images, annotated with circles and lines, respectively) [17], whereas edges detected on a satellite and matched against its geometric model (bottom right, red outline) serve as features for estimating the satellite's pose relative to the camera [21].…”

Section: Algorithms For Vision-based Navigation In Orbitmentioning

confidence: 99%

“…Our in-house development and testing involves a number of CPUs ranging from small space-grade and embedded devices to big desktop/laptop platforms. The C/C++ benchmarks used are Harris corner detection (employing five 2-D convolutions [17,28]), hyperspectral searching (of prestored spectral signatures in a pixel-by-pixel fashion [82] by using metrics such as χ 2 , L 1 , L 2 , cf. [83]), Canny edge detection [27], and binocular visual odometry [17] (including distinct algorithms for feature detection, feature description, temporal and spatial matching, pose estimation [84], outlier removal, etc.).…”

Section: B Development and In-house Testing 1 Central Processing Unitmentioning

confidence: 99%

See 2 more Smart Citations

High-Performance Embedded Computing in Space: Evaluation of Platforms for Vision-Based Navigation

Lentaris¹,

Maragos²,

Stratakos³

et al. 2018

Journal of Aerospace Information Systems

Self Cite

View full text Add to dashboard Cite

Vision-based navigation has become increasingly important in a variety of space applications for enhancing autonomy and dependability. Future missions, such as active debris removal for remediating the low Earth orbit environment, will rely on novel high-performance avionics to support advanced image processing algorithms with substantial workloads. However, when designing new avionics architectures, constraints relating to the use of electronics in space present great challenges, further exacerbated by the need for significantly faster processing compared to conventional space-grade central processing units. With the long-term goal of designing highperformance embedded computers for space, in this paper, an extended study and tradeoff analysis of a diverse set of computing platforms and architectures (i.e., central processing units, multicore digital signal processors, graphics processing units, and field-programmable gate arrays) are performed with radiation-hardened and commercial offthe-shelf technology. Overall, the study involves more than 30 devices and 10 benchmarks, which are selected after exploring the algorithms and specifications required for vision-based navigation. The present analysis combines literature survey and in-house development/testing to derive a sizable consistent picture of all possible solutions. Among others, the results show that certain 28 nm system-on-chip devices perform faster than space-grade and embedded central processing units by 1-3 orders of magnitude, while consuming less than 10 W. Field-programmable gate array platforms provide the highest performance per watt ratio.

show abstract

Section: Algorithms For Vision-based Navigation In Orbitmentioning

confidence: 99%

Section: Algorithms For Vision-based Navigation In Orbitmentioning

confidence: 99%

Section: B Development and In-house Testing 1 Central Processing Unitmentioning

confidence: 99%

See 1 more Smart Citation

High-Performance Embedded Computing in Space: Evaluation of Platforms for Vision-Based Navigation

Lentaris¹,

Maragos²,

Stratakos³

et al. 2018

Journal of Aerospace Information Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, this approach requires an RGB-D camera, which is unavailable and not preferred on many low-power robot agents, such as drones and planetary rovers. Lentaris et al [16] proposed the utilization of an space-worthy Field Programmable Gate Array (FPGA) co-processor to accelerate visual odometry to one order of magnitude faster than the software utilized by contemporary Mars rovers. This approach requires a FPGA co-processor to be added to the robot itself, which increases its weight and cost.…”

Section: Literature Reviewmentioning

confidence: 99%

VETO: An Immersive Virtual Environment for Tele-Operation

et al. 2018

View full text Add to dashboard Cite

This work investigates an over-arching question: how can an immersive virtual environment be connected with its intelligent physical counterpart to allow for a more efficient man-machine collaboration. To this end, an immersive user interface for the purpose of robot tele-operation is designed. A large amount of sensory data is utilized to build models of the world and its inhabitants in a way that is intuitive to the operator and accurately represents the robot's real-world state and environment. The game client is capable of handling multiple users, much like a traditional multiplayer game, while visualizing multiple robotic agents operating within the real world. The proposed Virtual Environment for Tele-Operation (VETO) architecture is a tele-operation system that provides a feature-rich framework to implement robotic agents into an immersive end-user game interface. Game levels are generated dynamically on a Graphic Processing Unit or GPU-accelerated server based on real-world sensor data from the robotic agents. A set of user studies are conducted to validate the performance of the proposed architecture compared to traditional tele-robotic applications. The experimental results show significant improvements in both task completion time and task completion rate over traditional tools.

show abstract

“…Accurately localizing moving cameras from monocular videos [63] is capable of providing scene context for high-level video understanding tasks [30], e.g., behavior analysis, action recognition, and so on, and has wide potentials in many intelligent systems, e.g., robotics, autonomous vehicles, and intelligent helicopters. While GPS devices are popular in these applications, they can only provide camera positions of moderate accuracies [55] and are sensitive to environment changes.…”

Section: Introductionmentioning

confidence: 99%

Untitled

2018

TIST

View full text Add to dashboard Cite

This article presents a high-precision multi-modal approach for localizing moving cameras with monocular videos, which has wide potentials in many intelligent applications, including robotics, autonomous vehicles, and so on. Existing visual odometry methods often suffer from symmetric or repetitive scene patterns, e.g., windows on buildings or parking stalls. To address this issue, we introduce a robust camera localization method that contributes in two aspects. First, we formulate feature tracking, the critical step of visual odometry, as a hierarchical min-cost network flow optimization task, and we regularize the formula with flow constraints, cross-scale consistencies, and motion heuristics. The proposed regularized formula is capable of adaptively selecting distinctive features or feature combinations, which is more effective than traditional methods that detect and group repetitive patterns in a separate step. Second, we develop a joint formula for integrating dense visual odometry and sparse GPS readings in a common reference coordinate. The fusion process is guided with high-order statistics knowledge to suppress the impacts of noises, clusters, and model drifting. We evaluate the proposed camera localization method on both public video datasets and a newly created dataset that includes scenes full of repetitive patterns. Results with comparisons show that our method can achieve comparable performance to state-of-the-art methods and is particularly effective for addressing repetitive pattern issues.

show abstract

HW/SW Codesign and FPGA Acceleration of Visual Odometry Algorithms for Rover Navigation on Mars

Cited by 43 publications

References 34 publications

High-Performance Embedded Computing in Space: Evaluation of Platforms for Vision-Based Navigation

High-Performance Embedded Computing in Space: Evaluation of Platforms for Vision-Based Navigation

VETO: An Immersive Virtual Environment for Tele-Operation

Untitled

Contact Info

Product

Resources

About