A hybrid motion estimation technique for fisheye video sequences based on equisolid re-projection

Abstract

Capturing large fields of view with only one camera is an important aspect in surveillance and automotive applications, but the wide-angle fisheye imagery thus obtained exhibits very special characteristics that may not be very well suited for typical image and video processing methods such as motion estimation. This paper introduces a motion estimation method that adapts to the typical radial characteristics of fisheye video sequences by making use of an equisolid reprojection after moving part of the motion vector search into the perspective domain via a corresponding back-projection. By combining this approach with conventional translational motion estimation and compensation, average gains in luminance PSNR of up to 1.14 dB are achieved for synthetic fisheye sequences and up to 0.96 dB for real-world data. Maximum gains for selected frame pairs amount to 2.40 dB and 1.39 dB for synthetic and real-world data, respectively.

A data set providing synthetic and real-world fisheye video sequences

“…With fisheye video sequences, however, the translational motion model is no longer valid. In [13], we therefore introduced a hybrid ME technique adapted to equisolid fisheye data which introduces suitable projections based on the equisolid fisheye model so as to be able to exploit the translational model while at the same time taking the fisheye characteristics into consideration. A small subset of the synthetic sequences presented here has already been used for evaluating this method.…”

“…The search range and block size employed here were 64 and 16 pixels, respectively. Details about the algorithm as well as further settings may be taken directly from [13]. With average obtainable gains of up to 2 dB, the need for fisheye adaptations is clearly given and thus, our proposed data set aims at helping develop these adaptations.…”

“…Because of this apparent lack of suitable fisheye material, a data set first created by the authors during their previous work on motion estimation and temporal error concealment for fisheye video sequences [13,14], and significantly extended since, is introduced in this paper and made freely available via their website. Both synthetically generated and actually captured fisheye video sequences are provided, ranging from simple patterns to more complex scenes, including automotive and surveillance content.…”

“…Making use of the synthetic sequences especially facilitates the verification and evaluation of new algorithms, since here, all parameters are known and an exact projection function is used so that calibration is unnecessary. To demonstrate the use of the data set and to show a practical application for our sequences, we revisit our previously published hybrid motion estimation method for fisheye Varying for each frame video sequences [13] and provide some new results for the synthetic sequences at the end of the paper.…”

Deformable block-based motion estimation in omnidirectional image sequences

“…Finally, block matching algorithms adapted to the geometry of fish-eye and catadioptric images have been proposed in [9], [10], [11], [12]. Due to the specificity of the considered acquisition systems, these algorithms are however not directly applicable to video sequences acquired with fully omnidirectional multi-dioptric camera systems available nowadays.…”