Iterative hierarchical true motion estimation for temporal frame interpolation

Abstract

Temporal frame interpolation is an important problem in many areas of modern video processing. However the task of accurate motion estimation for temporal interpolation is an open issue. In this paper we propose a new motion estimation algorithm for motion compensated frame interpolation. The developed algorithm is consistent with the conventional model of true motion and can be considered as a method of local minimization for the optimization problem defined within this model. The evaluation of the algorithm is performed for frame rate upconversion problem. Simulation results demonstrate performance comparable to existing frame rate up-conversion methods.

Occlusion-aware temporal frame interpolation in a highly scalable video coding setting

“…Veselov and Gilmutdinov [7] is about 10 times faster than the proposed method, while our BOA-TFI is around three times faster than Jeong et al [6].…”

“…The first row shows the full frame. The second to last row show crops of the ground truth, proposed BOA-TFI, Jeong et al [6] and Veselov and Gilmutdinov [7], respectively.…”

“…In this work, we significantly enhance the evaluation of the proposed method both qualitatively and quantitatively on various high-resolution natural sequences (Section VIII.A), and compare our performance with two state-of-the-art TFI methods [6,7]. One key distinguishing feature of the proposed method is its ability to handle regions around moving objects.…”

“…They show improved reconstruction quality, at 2 dominic rüfenacht, reji mathew and david taubman the expense of a significant increase in computational complexity. Veselov and Gilmutdinov [7] propose a hierarchical bidirectional multi-stage ME algorithm. They partition the target frame into non-overlapping, hierarchical blocks, and approximate the "true" motion flow.…”

Virtual Reality-Wireless Local Area Network: Wireless Connection-Oriented Virtual Reality Architecture for Next-Generation Virtual Reality Devices

“…Since both the technical progress of frame-interpolation schemes [11][12][13][14][15][16] and the frame-interpolation module are expected to be commonly employed in next-generation VR headsets [17], users could have stable high-refresh-rate vision with a lower downlink (DL) VR video frame rate. Sensing data, which is not fed back to computing machines, including VR-ready PCs, can be utilized by the frame-interpolation module to generate interpolation frames, with accurate moving data from the user.…”