An H.264 Sensor Aided Encoder for Aerial Video Sequences with In-the-Loop Metadata Enhancement

Abstract

Unmanned Aerial Vehicles (UAVs) are often employed to capture high resolution images in order to perform image mosaicking and/or 3D reconstruction. Images are usually stored on-board or sent to the ground using still image or video data compression. Still image encoders are preferred when low frame rates are involved, because video coding systems are based on motion estimation and compensation algorithms which fail when the motion vectors are significantly long. The latter is the case of low frame rate videos, in which the overlapping between subsequent frames is very small. In this scenario, UAVs attitude and position metadata from the Inertial Navigation System (INS) can be employed to estimate global motion parameters without video analysis. However, a low complexity analysis can refine the motion field estimated using only the metadata. In this work, we propose to use this refinement step in order to improve the position and attitude estimation produced by the navigation system with the aim of maximizing the encoder performance. Experiments on both simulated and real world video sequences confirm the effectiveness of the proposed approach.