Deep Iterative Frame Interpolation for Full-frame Video Stabilization

Choi, Jinsoo; Kweon, In So

doi:10.1145/3363550

Cited by 49 publications

(86 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6, the comparison with offline methods [7,8,12,13,17,18], online (real-time) methods [19,20,29,30], and ours indicates that the proposed method shows favorable performance for the majority of videos. In particular, our real-time method shows comparable cropping ratio to the recent approach proposed by Choi and Kweon [4], although their approach runs offline and is designed to prevent any cropping.…”

Section: Methodsmentioning

confidence: 88%

“…We also conduct comparisons with a widely used commercial product, Adobe Premiere (Pro CC 2017), and two recent approaches by Choi and Kweon [4] and by Yu and Ramanmoorthi [32]. Please refer to the supplementary material.…”

Section: Methodsmentioning

confidence: 99%

“…This work shows that spatially smoothed flow is useful for stabilizing frames, which our approach builds upon. Recently, Choi and Kweon [4] introduce an iterative frame interpolation approach to video stabilization.…”

Section: Related Workmentioning

confidence: 99%

“…This distinguishes our approach with supervised methods [29,30]. We carefully design the training methods for each module 4 , which is a crucial aspect of making our approach work. An overview of the loss functions is depicted in Fig.…”

Section: Self-supervised Learningmentioning

confidence: 99%

See 3 more Smart Citations

Self-Supervised Real-time Video Stabilization

Choi¹,

Park²,

Kweon³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Videos are a popular media form, where online video streaming has recently gathered much popularity. In this work, we propose a novel method of real-time video stabilization -transforming a shaky video to a stabilized video as if it were stabilized via gimbals in real-time. Our framework is trainable in a self-supervised manner, which does not require data captured with special hardware setups (i.e., two cameras on a stereo rig or additional motion sensors). Our framework consists of a transformation estimator between given frames for global stability adjustments, followed by scene parallax reduction module via spatially smoothed optical flow for further stability. Then, a margin inpainting module fills in the missing margin regions created during stabilization to reduce the amount of post-cropping. These sequential steps reduce distortion and margin cropping to a minimum while enhancing stability. Hence, our approach outperforms state-of-theart real-time video stabilization methods as well as offline methods that require camera trajectory optimization. Our method procedure takes approximately 24.3 ms yielding 41 fps regardless of resolution (e.g., 480p or 1080p).

show abstract

Section: Methodsmentioning

confidence: 88%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Self-supervised Learningmentioning

confidence: 99%

See 2 more Smart Citations

Self-Supervised Real-time Video Stabilization

Choi¹,

Park²,

Kweon³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…To remove jitters and generate stable video that can be viewed comfortably, many software video stabilization alogrithms have been proposed. Offline methods [1,2,3,4] are designed to stabilize videos that have been fully recorded. It usually requires a considerable amount of time to achieve an optimal result.…”

Section: Introductionmentioning

confidence: 99%

PixStabNet: Fast Multi-Scale Deep Online Video Stabilization with Pixel-Based Warping

Chen

Tseng

Lee

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Online video stabilizaton is increasingly needed for real-time applications such as live streaming, drone remote control, and video communication. We propose a multi-scale convolutional neural network (PixStabNet) which stabilizes video in real time without using future frames. Instead of calculating a global homography or multiple homographies, we estimate a pixel-based warping map to make the transformation of each pixel to achieve more precise modelling. In addition, we propose well-designed loss functions along with a two-stage training scheme to enhance network robustness. The quantitative result shows that our method outperforms other learningbased online methods in terms of stability with excellent geometric and temporal consistency. Moreover, to the best of our knowledge, the proposed algorithm is the most efficient approach for video stabilization. The models and results are available at: https://yu-ta-chen.github.io/PixStabNet.

show abstract

Attentive Cascaded Pyramid Network for Online Video Stabilization

Zhang

et al. 2022

Artificial Intelligence

View full text Add to dashboard Cite

Deep Iterative Frame Interpolation for Full-frame Video Stabilization

Abstract: The technology development for event recognition/relational reasoning and learning knowledge based system for video understanding).

Cited by 49 publications

References 30 publications

Self-Supervised Real-time Video Stabilization

Self-Supervised Real-time Video Stabilization

PixStabNet: Fast Multi-Scale Deep Online Video Stabilization with Pixel-Based Warping

Attentive Cascaded Pyramid Network for Online Video Stabilization

Contact Info

Product

Resources

About