Motion estimation for fisheye video sequences combining perspective projection with camera calibration information

Eichenseer, Andrea; Bätz, Michel; Kaup, André

doi:10.1109/icip.2016.7533210

Cited by 10 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, when dealing with an FOV of more than 180 • , we have to consider the peripheral fisheye regions where θ exceeds 90 • separately. To that end, we introduce an ultra wide-angle compensation procedure which handles the problem of faulty mappings [29]. Before discussing this proposed extension in detail, we first define r max and r 180 • , which are obtained by solving the equisolid fisheye projection function ( 5) for θ = FOV/2•π/180 and θ = π/2, respectively.…”

Section: Ultra Wide-angle Compensationmentioning

confidence: 99%

Motion Estimation for Fisheye Video With an Application to Temporal Resolution Enhancement

Eichenseer

Bätz

Kaup

2019

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

Surveying wide areas with only one camera is a typical scenario in surveillance and automotive applications. Ultra wide-angle fisheye cameras employed to that end produce video data with characteristics that differ significantly from conventional rectilinear imagery as obtained by perspective pinhole cameras. Those characteristics are not considered in typical image and video processing algorithms such as motion estimation, where translation is assumed to be the predominant kind of motion. This contribution introduces an adapted technique for use in block-based motion estimation that takes into account the projection function of fisheye cameras and thus compensates for the non-perspective properties of fisheye videos. By including suitable projections, the translational motion model that would otherwise only hold for perspective material is exploited, leading to improved motion estimation results without altering the source material. In addition, we discuss extensions that allow for a better prediction of the peripheral image areas, where motion estimation falters due to spatial constraints, and further include calibration information to account for lens properties deviating from the theoretical function. Simulations and experiments are conducted on synthetic as well as real-world fisheye video sequences that are part of a data set created in the context of this work. Average synthetic and real-world gains of 1.45 and 1.51 dB in luminance PSNR are achieved compared against conventional block matching. Furthermore, the proposed fisheye motion estimation method is successfully applied to motion compensated temporal resolution enhancement, where average gains amount to 0.79 and 0.76 dB.

show abstract

Section: Ultra Wide-angle Compensationmentioning

confidence: 99%

Motion Estimation for Fisheye Video With an Application to Temporal Resolution Enhancement

Eichenseer

Bätz

Kaup

2019

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moreover to the RGB or RGB-D image space, there also have been some researches about motion estimation for unique image space such as fisheye video sequence [16], and omnidirectional image space [17]. Eichenseer et al [16] proposed a hybrid block-based motion estimation method for real-world fisheye videos by combining perspective projection with considerations about the fisheye image structure. Simone et al [17] presented an extension of block-based motion estimation for panoramic omnidirectional image sequences by considering spherical geometry of the imaging system.…”

Section: Related Workmentioning

confidence: 99%

“…In this paper, to the best of our knowledge, we tackle the problem of ego-motion classification for compound eye cameras using convolutional neural network (CNN) for the first time, of which the goal is to classify the motion of compound eye camera given two consecutive emulated high resolution compound images. The ego-motion classification is the problem that focuses on the movement of the camera itself, which is different from the previous camera motion estimating algorithms that aim to estimate the transition between two scenes [12,13,14,15,16,17]. Here, the ego-motion classification of the compound eye camera gives an important contribution to the robot community, since knowing the moving direction of a robot is critical for problems such as localization and navigation.…”

Section: Introductionmentioning

confidence: 99%

Deep Ego-Motion Classifiers for Compound Eye Cameras

Yoo

Cha

2019

Sensors

View full text Add to dashboard Cite

Compound eyes, also known as insect eyes, have a unique structure. They have a hemispheric surface, and a lot of single eyes are deployed regularly on the surface. Thanks to this unique form, using the compound images has several advantages, such as a large field of view (FOV) with low aberrations. We can exploit these benefits in high-level vision applications, such as object recognition, or semantic segmentation for a moving robot, by emulating the compound images that describe the captured scenes from compound eye cameras. In this paper, to the best of our knowledge, we propose the first convolutional neural network (CNN)-based ego-motion classification algorithm designed for the compound eye structure. To achieve this, we introduce a voting-based approach that fully utilizes one of the unique features of compound images, specifically, the compound images consist of a lot of single eye images. The proposed method classifies a number of local motions by CNN, and these local classifications which represent the motions of each single eye image, are aggregated to the final classification by a voting procedure. For the experiments, we collected a new dataset for compound eye camera ego-motion classification which contains scenes of the inside and outside of a certain building. The samples of the proposed dataset consist of two consequent emulated compound images and the corresponding ego-motion class. The experimental results show that the proposed method has achieved the classification accuracy of 85.0%, which is superior compared to the baselines on the proposed dataset. Also, the proposed model is light-weight compared to the conventional CNN-based image recognition algorithms such as AlexNet, ResNet50, and MobileNetV2.

show abstract

“…Besides, Ahmmed et al [8] introduce the elastic model into HEVC to better describe the object motions in the fish eye cameras. Moreover, Eichenseer et al [9] propose a re-mapping method to handle some extreme cases in which the field of view is larger than 180-degrees. However, these motion models focusing on the fish eye cameras are unable to efficiently handle the cube map for 360-degree videos.…”

Section: Introductionmentioning

confidence: 99%

Projection based advanced motion model for cubic mapping for 360-degree video

Budagavi

et al. 2017

2017 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

This paper proposes a novel advanced motion model to handle the irregular motion for the cubic map projection of 360degree video. Since the irregular motion is mainly caused by the projection from the sphere to the cube map, we first try to project the pixels in both the current picture and reference picture from unfolding cube back to the sphere. Then through utilizing the characteristic that most of the motions in the sphere are uniform, we can derive the relationship between the motion vectors of various pixels in the unfold cube. The proposed advanced motion model is implemented in the High Efficiency Video Coding reference software. Experimental results demonstrate that quite obvious performance improvement can be achieved for the sequences with obvious motions.

show abstract

Motion estimation for fisheye video sequences combining perspective projection with camera calibration information

Cited by 10 publications

References 18 publications

Motion Estimation for Fisheye Video With an Application to Temporal Resolution Enhancement

Motion Estimation for Fisheye Video With an Application to Temporal Resolution Enhancement

Deep Ego-Motion Classifiers for Compound Eye Cameras

Projection based advanced motion model for cubic mapping for 360-degree video

Contact Info

Product

Resources

About