Temporally Consistent Gradient Domain Video Editing

Abstract

In the context of video editing, enforcing spatio-temporal consistency is an important issue. With that purpose, the current variational models for gradient domain video editing include space and time regularization terms. The spatial terms are based on the usual space derivatives, the temporal ones are based on the convective derivative, and both are balanced by a parameter β. However, the usual discretizations of the convective derivative limit the value of β to a certain range, thus limiting these models from achieving their full potential. In this paper, we propose a new numerical scheme to compute the convective derivative, the deblurring convective derivative, which allows us to lift this constraint. Moreover, the proposed scheme introduces less errors than other discretization schemes without adding computational complexity. We use this scheme in the implementation of two gradient domain models for temporally consistent video editing, based on Poisson and total variation type formulations, respectively. We apply these models to three video editing tasks: inpainting correction, object insertion and object removal.

Towards fast, generic video inpainting

“…Another recent method by Facciolo et al [15] deals with video editing in the gradient domain. However, the goal of this method is to enforce temporal consistency given a certain inpainting result, rather than producing the inpainting result itself.…”

Spatio-temporal binary video inpainting via threshold dynamics

“…It includes the convective derivative and has no restrictions on background nor foreground movements. The convective derivative has been used for ensuring spatio-temporal consistency in other video editing applications [6,5,17,31].…”

Efficient Poisson Image Editing

“…It is the process of combining two images by copying the gradients of the pixels from the source image into the target image. It is used in many applications such as seamless cloning [14][15][16][17][18][19][20][21][22], seamless video editing [23][24], seamless stitching [25][26][27], shadow removal [28], inpainting [29][30][31], gradient domain painting [32,33], scene completion [34,35], and image smoothing [36][37][38]. Some of gradient domain methods compose images by solving the Poisson equation in the specified region.…”