IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation

Kong, Lingtong; Jiang, Boyuan; Luo, Donghao; Chu, Wenqing; Huang, Xiaoming; Tai, Ying; Wang, Chengjie; Yang, Jie

doi:10.48550/arxiv.2205.14620

Cited by 1 publication

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing mainstream methods of video frame interpolation can be generally categorized into flow-based methods [7][8][9]11,12,[15][16][17][18][19][20][21][22][23][24][25][26], kernel-based methods [1,5,10,[27][28][29][30], phase-based methods [31], and hallucination-based methods [2,3,14,32].…”

Section: Related Workmentioning

confidence: 99%

“…AllAtOnce [21] further extends quadratic motion to cubic motion. IFRNet [11] gradually estimates bilateral optical flow together with intermediate features in a coarse-to-fine manner for mutual promotion of the two components. VFIformer [23] refines the warped frames with a Transformer [34] structure.…”

Section: Related Workmentioning

confidence: 99%

“…For most flow-based methods, the bilateral optical flows between the intermediate frame and input frames can be simply obtained by scaling the optical flow between input frames by t [7][8][9]12]. IFRNet [11] takes t as input to the network to estimate optical flow. EDSC [10] is a kernelbased method using heterogeneous convolution [35] and deformable convolution [36] with separable filters.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, some researchers have focused on generating interpolated frames for arbitrary scales [7][8][9][10][11][12]. Most of these methods depend on scaling the estimated optical flow linearly according to the desired scale.…”

Section: Introductionmentioning

confidence: 99%

“…Second, the computation of each output frame only directly depends on (the features of) input frames, without dependency on any other output frames, and thus ATVFI is free from interpolation error accumulation. Compared to VFI methods [7][8][9]11,12], considering arbitrary interpolation scale, our ATVFI does not assume linear motion, and its motion estimation is accordingly learned from training samples.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Arbitrary Timestep Video Frame Interpolation with Time-Dependent Decoding

Zhang,

Ren,

Yan

et al. 2024

Mathematics

View full text Add to dashboard Cite

Given an observed low frame rate video, video frame interpolation (VFI) aims to generate a high frame rate video, which has smooth video frames with higher frames per second (FPS). Most existing VFI methods often focus on generating one frame at a specific timestep, e.g., 0.5, between every two frames, thus lacking the flexibility to increase the video’s FPS by an arbitrary scale, e.g., 3. To better address this issue, in this paper, we propose an arbitrary timestep video frame interpolation (ATVFI) network with time-dependent decoding. Generally, the proposed ATVFI is an encoder–decoder architecture, where the interpolation timestep is an extra input added to the decoder network; this enables ATVFI to interpolate frames at arbitrary timesteps between input frames and to increase the video’s FPS at any given scale. Moreover, we propose a data augmentation method, i.e., multi-width window sampling, where video frames can be split into training samples with multiple window widths, to better leverage training frames for arbitrary timestep interpolation. Extensive experiments were conducted to demonstrate the superiority of our model over existing baseline models on several testing datasets. Specifically, our model trained on the GoPro training set achieved 32.50 on the PSNR metric on the commonly used Vimeo90k testing set.

show abstract