Real-Time Image Stabilization Method Based on Optical Flow and Binary Point Feature Matching

Deng, Zilong; Yang, Daoyong; Zhang, Xiaohu; Dong, Yuguang; Liu, Chengbo; Shen, Qiang

doi:10.3390/electronics9010198

Cited by 12 publications

(4 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dealing with motion blur can improve the localization and map construction accuracy of vision-related SLAM systems. Following are a few common methods to deal with motion blurred image frames: motion compensation [26]: by analyzing the trajectory of moving objects in the image, the motion of the camera can be estimated and motion compensated in the image. This can be achieved by image alignment algorithms such as optical flow algorithms or feature point matching algorithms.…”

Section: Motion Blur Image Frame Processingmentioning

confidence: 99%

UMPL- VINS: Generalized SLAM with Point and Line Features for Multi-scene Applications

Jiang,

Shang,

Xue

et al. 2024

Preprint

View full text Add to dashboard Cite

The use of line features to improve the localization accuracy of visual inertial SLAM (VINS) based on point features is of increasing interest because of the additional constraints they provide on the scene structure. It is found that although the introduction of line features improves the accuracy of relative position estimation in some scenes due to the additional constraints, the two constraints only achieve an equalization for the estimated relative position. Therefore, in some environments two constraints can reduce the accuracy of the single-feature constraint algorithm and also make the real-time performance of the system more challenging. To address such issues, in this paper, we design a generalized SLAM system with point-line features that can be applied to multiple scenarios. We first enhance the image frames used for feature extraction by eliminating motion blur frames through the fuzzy metric method and mutation modeling of velocity and rotation. Then we improve the traditional line detection model by short line fusion, uniform distribution of line features, and refinement of edge features to obtain high-quality line features for building SLAM. Finally, based on the application of point features and line features for different scenes, a point and line feature separation-union model is proposed. In addition, we design a transformation model for line-point features to enhance the processing of point-line features. Experimental results on EuRoc, TUM_VI, KITTI, PennCOSYVIO and our own recorded dataset prove that the method proposed in this paper realizes a generalized SLAM with point and line features applied to multi-scenes with good advantages.

show abstract

Section: Motion Blur Image Frame Processingmentioning

confidence: 99%

UMPL- VINS: Generalized SLAM with Point and Line Features for Multi-scene Applications

Jiang,

Shang,

Xue

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Thus, tracking static features relative to their position in a reference frame constitutes the basis of frame-to-reference stabilisation (Morimoto and Chellappa, 1996b). To guarantee that good features to track are present and well-distributed in the ROI, UAS velocimetry commonly employs arrays of artificial ground control points (GCPs) which are used as static features for stabilisation (Detert et al, 2017;Detert and Weitbrecht, 2014;Perks et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

A comparison of tools and techniques for stabilising unmanned aerial system (UAS) imagery for surface flow observations

Ljubičić

Strelnikova

Perks

et al. 2021

Hydrol. Earth Syst. Sci.

View full text Add to dashboard Cite

Abstract. While the availability and affordability of unmanned aerial systems (UASs) has led to the rapid development of remote sensing applications in hydrology and hydrometry, uncertainties related to such measurements must be quantified and mitigated. The physical instability of the UAS platform inevitably induces motion in the acquired videos and can have a significant impact on the accuracy of camera-based measurements, such as velocimetry. A common practice in data preprocessing is compensation of platform-induced motion by means of digital image stabilisation (DIS) methods, which use the visual information from the captured videos – in the form of static features – to first estimate and then compensate for such motion. Most existing stabilisation approaches rely either on customised tools developed in-house, based on different algorithms, or on general purpose commercial software. Intercomparison of different stabilisation tools for UAS remote sensing purposes that could serve as a basis for selecting a particular tool in given conditions has not been found in the literature. In this paper, we have attempted to summarise and describe several freely available DIS tools applicable to UAS velocimetry. A total of seven tools – six aimed specifically at velocimetry and one general purpose software – were investigated in terms of their (1) stabilisation accuracy in various conditions, (2) robustness, (3) computational complexity, and (4) user experience, using three case study videos with different flight and ground conditions. In an attempt to adequately quantify the accuracy of the stabilisation using different tools, we have also presented a comparison metric based on root mean squared differences (RMSDs) of inter-frame pixel intensities for selected static features. The most apparent differences between the investigated tools have been found with regards to the method for identifying static features in videos, i.e. manual selection of features or automatic. State-of-the-art methods which rely on automatic selection of features require fewer user-provided parameters and are able to select a significantly higher number of potentially static features (by several orders of magnitude) when compared to the methods which require manual identification of such features. This allows the former to achieve a higher stabilisation accuracy, but manual feature selection methods have demonstrated lower computational complexity and better robustness in complex field conditions. While this paper does not intend to identify the optimal stabilisation tool for UAS-based velocimetry purposes, it does aim to shed light on details of implementation, which can help engineers and researchers choose the tool suitable for their needs and specific field conditions. Additionally, the RMSD comparison metric presented in this paper can be used in order to measure the velocity estimation uncertainty induced by UAS motion.

show abstract

“…Image registration plays an important role in computer vision. Image registration is widely used in many aspects such as image matching [1][2][3][4][5][6][7], change detection [8,9], 3D reconstruction [10][11][12], guidance [13][14][15], mapping sciences [16][17][18][19][20][21], and mobile robot [22,23]. In general, image registration methods can be mainly divided into two kinds: gray-scale matching methods and feature-based matching methods.…”

Section: Introductionmentioning

confidence: 99%

Homogenized ORB Algorithm Using Dynamic Threshold and Improved Quadtree

Xiao

2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

The Oriented FAST and Rotated BRIEF (ORB) algorithm has the problem that the extracted feature points are overconcentrated or even overlapped, leading to information loss of local image features. A homogenized ORB algorithm using dynamic thresholds and improved quadtree method is proposed in this paper, named Quadtree ORB (QTORB). In the feature point extraction stage, a new dynamic local threshold calculation method is proposed to enhance the algorithm’s ability to extract feature points at homogeneous regions. Then, a quadtree method is improved and adopted to manage and optimize feature points to eliminate those excessively concentrated and overlapping feature points. Meanwhile, in the feature points optimization process, different quadtree depths are set at different image pyramid levels to prevent excessive splitting of the quadtree and increase calculation speed. In the feature point description stage, local gray difference value information is introduced to enhance the saliency of the feature description. Finally, the Hamming distance is used to match points and RANSAC is used to avoid mismatches. Two datasets, namely, the optical image dataset and SAR image dataset, are used in the experiment. The experimental result shows that, considering accuracy and real-time efficiency, the QTORB can effectively improve the distribution uniformity of feature points.

show abstract

Real-Time Image Stabilization Method Based on Optical Flow and Binary Point Feature Matching

Cited by 12 publications

References 27 publications

UMPL- VINS: Generalized SLAM with Point and Line Features for Multi-scene Applications

UMPL- VINS: Generalized SLAM with Point and Line Features for Multi-scene Applications

A comparison of tools and techniques for stabilising unmanned aerial system (UAS) imagery for surface flow observations

Homogenized ORB Algorithm Using Dynamic Threshold and Improved Quadtree

Contact Info

Product

Resources

About