Horn-Schunck Optical Flow with a Multi-Scale Strategy

Abstract

The seminal work of Horn and Schunck is the first variational method for optical flow estimation. It introduced a novel framework where the optical flow is computed as the solution of a minimization problem. From the assumption that pixel intensities do not change over time, the optical flow constraint equation is derived. This equation relates the optical flow with the derivatives of the image. There are infinitely many vector fields that satisfy the optical flow constraint, thus the problem is ill-posed. To overcome this problem, Horn and Schunck introduced an additional regularity condition that restricts the possible solutions. Their method minimizes both the optical flow constraint and the magnitude of the variations of the flow field, producing smooth vector fields. One of the limitations of this method is that, typically, it can only estimate small motions. In the presence of large displacements, this method fails when the gradient of the image is not smooth enough. In this work, we describe an implementation of the original Horn and Schunck method and also introduce a multi-scale strategy in order to deal with larger displacements. For this multi-scale strategy, we create a pyramidal structure of downsampled images and change the optical flow constraint equation with a nonlinear formulation. In order to tackle this nonlinear formula, we linearize it and solve the method iteratively in each scale. In this sense, there are two common approaches: one that computes the motion increment in the iterations; or the one we follow, that computes the full flow during the iterations. The solutions are incrementally refined over the scales. This pyramidal structure is a standard tool in many optical flow methods. Source Code A standalone ANSI C implementation is available 1. This file contains two main programs: horn schunck classic.c, which implements the original Horn and Schunck method; and the implementation of the multi-scale approach, in file horn schunck pyramidal.c. This latter implementation is best suited for general image sequences.

The Inverse Compositional Algorithm for Parametric Registration

“…In order to estimate large displacements, we use a pyramidal structure. We follow the same strategy presented in previous IPOL articles [16,14,17] and reproduce here the basic ideas. Our algorithm creates a pyramid of down-sampled images.…”

“…The value of σ 0 is a small constant to reduce the noise of the original image and is found empirically, as explained in [14]. The motion is normally initialized as p := (0, 0) and the system of equations is solved at the coarsest scale.…”

Robust Discontinuity Preserving Optical Flow Methods

“…We follow the same strategy presented in the IPOL articles [9] and [16]. Before downsampling, the algorithm smoothes the images with a Gaussian kernel of a standard deviation that depends on σ ∈ (0, 1).…”

“…Many strategies have been proposed to cope with motion discontinuities since the method of Horn and Schunck [8]. This technique hardly preserves flow boundaries as can be seen in the IPOL article by Meinhardt-Llopis et al [9].…”

An Implementation of Combined Local-Global Optical Flow

“…This can be a problem for image sequences with large object displacements that can appear as discontinuities and lead to erroneous solutions. To address this issue, multiscale (MS) strategies have been proposed [4,11], using a coarse-to-fine approach to successively compute more accurate OF vector fields.…”