TriangleFlow: Optical Flow with Triangulation-Based Higher-Order Likelihoods

Abstract

We use a simple yet powerful higher-order conditional random field (CRF) to model optical flow. It consists of a standard photoconsistency cost and a prior on affine motions both modeled in terms of higher-order potential functions. Reasoning jointly over a large set of unknown variables provides more reliable motion estimates and a robust matching criterion. One of the main contributions is that unlike previous region-based methods, we omit the assumption of constant flow. Instead, we consider local affine warps whose likelihood energy can be computed exactly without approximations. This results in a tractable, so-called, higher-order likelihood function. We realize this idea by employing triangulation meshes which immensely reduce the complexity of the problem. Optimization is performed by hierarchical fusion moves and an adaptive mesh refinement strategy. Experiments show that we achieve high-quality motion fields on several data sets including the Middlebury optical flow database.

Optical flow modeling and computation: A survey

“…For more details about existing move-spaces, see [249,255]. Applications for optical flow have increased rapidly in recent years [59,64,88,101,99,161,164,234,279].…”

Deformable image alignment as a source of stereo correspondences on portraits

“…[16,20]. Some approaches, however, also employ triangle meshes, for example the discrete algorithm by [9] or the finite element approach of [5]. Optical flow and MDIA algorithms often share the brightness constancy assumption, the violation of which severely hampers results.…”

Higher-Order Clique Reduction without Auxiliary Variables

“…Although their importance was long understood and there were earlier attempts on utilizing them [5,18,21], their use [9,27] and research into minimization of various classes of higherorder MRF energies [1,6,8,11,12,17,20,24,25,26] have intensified significantly in the past few years. While there are useful higher-order MRF energies with specific forms that can be efficiently minimized [15,16], minimization of general higher-order energy is needed to utilize sophisticated priors, especially if it is learned from data as in the case of Fields of Experts (FoEs) [23].…”