FPGA-based ORB feature extraction for real-time visual SLAM

Fang, Weikang; Zhang, Yanjun; Bo, Yu; Liu, Shaoshan

doi:10.1109/fpt.2017.8280159

Cited by 63 publications

(30 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with the ORB extractor implemented on FPGA in [4], the ORB extractor in eSLAM has deployed hardware-friendly optimization, such as RS-BRIEF and workflow rescheduling. Hence, the latency of feature extraction in eSLAM is approximately 39% less than the latency of [4], even if 48% more pixels are processed in eSLAM because of the involved extra two layers in the image pyramid. Table 3: Frame rate and energy efficiency comparison results, where "N-frame" represents the normal frame, and "Kframe" represents the key frame.…”

Section: Discussionmentioning

confidence: 99%

“…Several prior efforts have been made to accelerate visual SLAM on low-power platforms, but no fully integrated ORB-based visual SLAM is proposed on such platforms so far. Feature matching and ORB extraction is accelerated on FPGA for visual SLAM system, respectively in [2] and [4]. A SIFT-feature based SLAM is implemented on FPGA [6] where only matrix computation is accelerated but the most time-consuming part, feature extraction, is not involved.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

eSLAM

Liu

Yang

Chen

et al. 2019

Proceedings of the 56th Annual Design Automation Conference 2019

View full text Add to dashboard Cite

Simultaneous Localization and Mapping (SLAM) is a critical task for autonomous navigation. However, due to the computational complexity of SLAM algorithms, it is very difficult to achieve realtime implementation on low-power platforms. We propose an energyefficient architecture for real-time ORB (Oriented-FAST and Rotated-BRIEF) based visual SLAM system by accelerating the most timeconsuming stages of feature extraction and matching on FPGA platform. Moreover, the original ORB descriptor pattern is reformed as a rotational symmetric manner which is much more hardware friendly. Optimizations including rescheduling and parallelizing are further utilized to improve the throughput and reduce the memory footprint. Compared with Intel i7 and ARM Cortex-A9 CPUs on TUM dataset, our FPGA realization achieves up to 3× and 31× frame rate improvement, as well as up to 71× and 25× energy efficiency improvement, respectively.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

eSLAM

Liu

Yang

Chen

et al. 2019

Proceedings of the 56th Annual Design Automation Conference 2019

View full text Add to dashboard Cite

show abstract

“…A parallel hardware architecture for SIFT was also reported in Reference [34]. Recent and improved FPGA implementations for SURF [35][36][37] and ORB [38,39] were reported. SIFT and SURF implementations were not included in our comparison because they were intensity-based not binary descriptors.…”

Section: Comparison With Other Implementationsmentioning

confidence: 99%

Optimization and Implementation of Synthetic Basis Feature Descriptor on FPGA

2020

View full text Add to dashboard Cite

Feature detection, description, and matching are crucial steps for many computer vision algorithms. These steps rely on feature descriptors to match image features across sets of images. Previous work has shown that our SYnthetic BAsis (SYBA) feature descriptor can offer superior performance to other binary descriptors. This paper focused on various optimizations and hardware implementation of the newer and optimized version. The hardware implementation on a field-programmable gate array (FPGA) is a high-throughput low-latency solution which is critical for applications such as high-speed object detection and tracking, stereo vision, visual odometry, structure from motion, and optical flow. We compared our solution to other hardware designs of binary descriptors. We demonstrated that our implementation of SYBA as a feature descriptor in hardware offered superior image feature matching performance and used fewer resources than most binary feature descriptor implementations.

show abstract

“…The matching quality of the image pairs will directly affect the autonomous navigation and modeling quality of the subsequent mobile platform. To improve the performance of the ORB algorithm, researchers have mainly applied the following approaches: (1) improving the feature point extraction strategy [7], [13]- [19], (2) enhancing the feature descriptor [20]- [22], and (3) upgrading the strategy of matching feature points [23]- [28]. For example, to improve the distribution uniformity of feature points, enhance the matching efficiency, and increase the correct matching rate, the ORB-slam2 [29] initial matching algorithm boosts the feature point matching efficiency; this improvement is achieved by rasterizing the entire image and modifying the minimum Hamming matching distance to 0.7-0.9 times the sub-optimal matching distance; moreover, the histogram statistics are used to describe the sub-rotation main direction angle, and only the top three direction feature points are selected in the statistics to improve the correct matching rate.…”

Section: Introductionmentioning

confidence: 99%

Multi-Stage Matching Approach for Mobile Platform Visual Imagery

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Unpredictable texture structure and motion blur continuously exist in mobile platform visual imagery and seriously reduce the similarity between images. Thus, accurate, stable, and well-distributed matches to follow the accurate pose estimation of the platform are difficult to obtain. To solve such problems, an effective image matching method for mobile platform visual imagery is presented in this study. The proposed method includes three steps, namely, standard initial matching, transformation matrices evaluation and matching propagation. Firstly, an oriented FAST and rotated BRIEF (ORB) method was used to obtain the number of matches and the initial projective transformation relationship between an image pair. Secondly, an evaluation function was set to choose the suitable rotation matrix for the image scene. Finally, geometric correspondence matching was utilized to propagate matches and produce additional reliable matching results. The geometric correspondence matching used the geometric relationship between the image pair and found more suitable matches than the standard ORB matching. Comprehensive experiments on TUM and ICL-NUIM dataset images showed that the proposed algorithm performs better in terms of correct matches, satisfactory matching rate, and higher matching accuracy than the standard ORB and ORB-slam2 initial match methods. INDEX TERMS Mobile platform visual imagery, image matching, ORB, propagation matching, geometric correspondence matching.

show abstract

FPGA-based ORB feature extraction for real-time visual SLAM

Cited by 63 publications

References 10 publications

eSLAM

eSLAM

Optimization and Implementation of Synthetic Basis Feature Descriptor on FPGA

Multi-Stage Matching Approach for Mobile Platform Visual Imagery

Contact Info

Product

Resources

About