Video Frame Interpolation via Deformable Separable Convolution

Cheng, Xianhang; Chen, Zhenzhong

doi:10.1609/aaai.v34i07.6634

Cited by 100 publications

(66 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e convolutional layer is the core of the network, and most calculations are performed in the convolutional layer. e feature map is generated in the convolution operation and output to the next layer for feature extraction [13]. In the convolution operation, the convolution kernel learns the best parameters for extracting features through iterations.…”

Section: Convolutional Layermentioning

confidence: 99%

Rethinking Separable Convolutional Encoders for End-to-End Semantic Image Segmentation

Wang

Hawbani

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

With the development of science and technology, the middle volume and neural network in the semantic image segmentation of the codec show good development prospects. Its advantage is that it can extract richer semantic features, but this will cause high costs. In order to solve this problem, this article mainly introduces the codec based on a separable convolutional neural network for semantic image segmentation. This article proposes a codec based on a separable convolutional neural network for semantic image segmentation research methods, including the traditional convolutional neural network hierarchy into a separable convolutional neural network, which can reduce the cost of image data segmentation and improve processing efficiency. Moreover, this article builds a separable convolutional neural network codec structure and designs a semantic segmentation process, so that the codec based on a separable convolutional neural network is used for semantic image segmentation research experiments. The experimental results show that the average improvement of the dataset by the improved codec is 0.01, which proves the effectiveness of the improved SegProNet. The smaller the number of training set samples, the more obvious the performance improvement.

show abstract

Section: Convolutional Layermentioning

confidence: 99%

Rethinking Separable Convolutional Encoders for End-to-End Semantic Image Segmentation

Wang

Hawbani

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…Motion approximation for backward warping: Conventional algorithms [2,3,16,34,36] approximate the motion fields V t→0 and V t→1 in (6). For example, the flow projection in [2,3] approximates V t→0 and V t→1 by aggregating multiple flow vectors between I 0 and I 1 , which pass near each pixel in I t .…”

Section: Motion-based Frame Warpingmentioning

confidence: 99%

BMBC: Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation

Park¹,

Ko²,

Lee

et al. 2020

Lecture Notes in Computer Science

144

128

View full text Add to dashboard Cite

We propose a novel video frame interpolation algorithm based on asymmetric bilateral motion estimation (ABME), which synthesizes an intermediate frame between two input frames. First, we predict symmetric bilateral motion fields to interpolate an anchor frame. Second, we estimate asymmetric bilateral motions fields from the anchor frame to the input frames. Third, we use the asymmetric fields to warp the input frames backward and reconstruct the intermediate frame. Last, to refine the intermediate frame, we develop a new synthesis network that generates a set of dynamic filters and a residual frame using local and global information. Experimental results show that the proposed algorithm achieves excellent performance on various datasets. The source codes and pretrained models are available at https://github.com/JunHeum/ABME.

show abstract

“…Thus, we adopt a lightweight optical flow network [31] on LR frames and a flow refine network [26] to get the middle flow on HR frames, and we try a new supervised flow loss to achieve better perception. Recently, meta-learning is also introduced into frame interpolation [7]; CAIN [8] adapts channel attention into VFI; and EDSC [6] uses ConvLSTM to learn motion offset for implicit motion compensation.…”

Section: Video Frame Interpolation (Vfi)mentioning

confidence: 99%

How Video Super-Resolution and Frame Interpolation Mutually Benefit

Zhou

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Video super-resolution (VSR) and video frame interpolation (VFI) are inter-dependent for enhancing videos of low resolution and low frame rate. However, most studies treat VSR and temporal VFI as independent tasks. In this work, we design a spatial-temporal superresolution network based on exploring the interaction between VSR and VFI. The main idea is to improve the middle frame of VFI by the super-resolution (SR) frames and feature maps from VSR. In the meantime, VFI also provides extra information for VSR and thus, through interacting, the SR of consecutive frames of the original video can also be improved by the feedback from the generated middle frame. Drawing on this, our approach leverages a simple interaction of VSR and VFI and achieves state-of-the-art performance on various datasets. Due to such a simple strategy, our approach is universally applicable to any existing VSR or VFI networks for effectively improving their video enhancement performance.

show abstract

Video Frame Interpolation via Deformable Separable Convolution

Cited by 100 publications

References 27 publications

Rethinking Separable Convolutional Encoders for End-to-End Semantic Image Segmentation

Rethinking Separable Convolutional Encoders for End-to-End Semantic Image Segmentation

BMBC: Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation

How Video Super-Resolution and Frame Interpolation Mutually Benefit

Contact Info

Product

Resources

About