Motion Inpainting by an Image-Based Geodesic AMLE Method

Abstract

This work presents an automatic method for optical flow inpainting. Given a video, each frame domain is endowed with a Riemannian metric based on the video pixel values. The missing optical flow is recovered by solving the Absolutely Minimizing Lipschitz Extension (AMLE) partial differential equation on the Riemannian manifold. An efficient numerical algorithm is proposed using eikonal operators for nonlinear elliptic partial differential equations on a finite graph. The choice of the metric is discussed and the method is applied to optical flow inpainting and sparse-to-dense optical flow estimation, achieving top-tier performance in terms of End-Point-Error (EPE).

On Anisotropic Optical Flow Inpainting Algorithms

“…The effect of these resolutions will be analyzed in Section 2.4. In [18], this scheme is applied for x ∈ Ω 0 keeping the known values of v 1 (x), respectively v 2 (x), on the known region Ω \ Ω 0 for all k, and initializing u k=0 (x) = 0 in Ω 0 .…”

“…A common solution is to inpaint or fill in the optical flow in such regions. In this work the problem of filling in an incomplete optical flow is addressed using anisotropic interpolation in Riemannian manifolds, as proposed in [18]. The interpolation is solved on a Riemannian manifold in order to take advantage of the geometric information given by the video frames: Given a video and an incomplete motion field, each 2D frame domain is endowed with a Riemannian metric based on the video values.…”

“…In this work a detailed analysis and description of [18] and a variant of it using the Laplace-Beltrami operator are provided. The rest of the paper is organized as follows.…”

SSGP: Sparse Spatial Guided Propagation for Robust and Generic Interpolation

“…The problems of interpolation and extrapolation have a long history in mathematics and computer science. In high-level computer vision, interpolation finds its application in various problems like motion estimation in 2D (optical flow) [1,2,11,14,15,20,30,31,33,43,50], 3D (scene flow) [36,37], or depth completion [6,17,26,40,41]. These methods in turn are applied in robot navigation, advanced driver assistance systems (ADAS), surveillance, and many others.…”