Reconstructing Optical Flow Fields by Motion Inpainting

Abstract

Abstract. An edge-sensitive variational approach for the restoration of optical flow fields is presented. Real world optical flow fields are frequently corrupted by noise, reflection artifacts or missing local information. Still, applications may require dense motion fields. In this paper, we pick up image inpainting methodology to restore motion fields, which have been extracted from image sequences based on a statistical hypothesis test on neighboring flow vectors. A motion field inpainting model is presented, which takes into account additional information from the image sequence to improve the reconstruction result. The underlying functional directly combines motion and image information and allows to control the impact of image edges on the motion field reconstruction. In fact, in case of jumps of the motion field, where the jump set coincides with an edge set of the underlying image intensity, an anisotropic TV-type functional acts as a prior in the inpainting model. We compare the resulting image guided motion inpainting algorithm to diffusion and standard TV inpainting methods.

Motion Inpainting by an Image-Based Geodesic AMLE Method

“…where y and z are the pixels providing the maximum and minimun in (6) and 7. This scheme is applied for x ∈ Ω 0 keeping the values of v 1 (x), respectively v 2 (x), on the known region Ω \ Ω 0 for all k. This scheme is embedded in a multiscale approach: the input optical flow and corresponding video frame are downscaled to a set of scales and the solution is computed at each one using (9).…”

“…Kondermann et al [13] proposed a postprocess of the optical flow in order to improve it: the optical flow is retained at points where it is reliable and is then densified by minimizing the L 2 norm of the spatio-temporal gradient of the flow. Berkels et al [6] proposed to recover the optical flow in non-reliable regions using a TV-type anisotropic functional and a rotation-invariant regularizer was proposed by Palomares et al [21]. On the other hand, Ince and Konrad [11] introduced a variational method for the joint estimation of optical flow and occlusions while extrapolating the optical flow in occluded areas by means of anisotropic diffusion based on the image gradients.…”

A Variational Aggregation Framework for Patch-Based Optical Flow Estimation

“…Exemplar-based image inpainting fills missing parts by copying pixels of the observed image. In motion estimation, occlusion filling is usually solved by diffusionbased (or geometry-oriented) schemes, propagating motion from non-occluded regions to occluded regions using partial derivative equation (PDE) resolution [3,9,47,51,64,86]. In contrast, we adopt an exemplar-based strategy for candidates computation in occluded regions.…”