Variational Optical Flow Estimation Based on Stick Tensor Voting

Abstract

Variational optical flow techniques allow the estimation of flow fields from spatio-temporal derivatives. They are based on minimizing a functional that contains a data term and a regularization term. Recently, numerous approaches have been presented for improving the accuracy of the estimated flow fields. Among them, tensor voting has been shown to be particularly effective in the preservation of flow discontinuities. This paper presents an adaptation of the data term by using anisotropic stick tensor voting in order to gain robustness against noise and outliers with significantly lower computational cost than (full) tensor voting. In addition, an anisotropic complementary smoothness term depending on directional information estimated through stick tensor voting is utilized in order to preserve discontinuity capabilities of the estimated flow fields. Finally, a weighted non-local term that depends on both the estimated directional information and the occlusion state of pixels is integrated during the optimization process in order to denoise the final flow field. The proposed approach yields state-of-the-art results on the Middlebury benchmark.

Optical Flow Estimation Combining Spatial-Temporal Derivatives Based Nonlinear Filtering

“…Still, STDF method performs worse on Hydrangea and Venus sequences because their flow fields are smooth. For the application scenarios of STDF and SIF are different, our future work aims at combining the two filters together to design a more effective filter used in optical flow estimation and improving the weighted non-local term in formula (8) in [7] based on our theory.…”

“…In order to simplify the comparison process in the latter section, we approximately regard the weighted non-local term [7], [8] as the following filter which is based on spatial distance and difference of intensities (SIF):…”

“…Moreover, median filter may lead the edges and corners to over-smoothing, which is undesirable for our purpose. Weighted non-local terms based on pixels' spatial distance and their dissimilarity of intensities are mentioned in [7], [8]. They assumed that pixels which have geometric closeness and photometric similarity have a similar velocity.…”

Spatially-Variant Kernel for Optical Flow Under Low Signal-to-Noise Ratios Application to Microscopy

“…To reduce the over-smoothing produced by the shift-invariant filtering of the data term in [6], the authors of [13] replaced the Gaussian filter by a bilateral filter, and [22] exploited tensor voting. These latter approaches rely on image measurements to specify filters.…”

“…If a discontinuity is contained in the Gaussian support defined by σ, the locally constant motion assumption will be violated and lower values σ will be favoured. Rather than being adapted to the image content as in [13,22], σ is guided now by the data term and variations follow motion discontinuities rather than image discontinuities. Note that the inverse barrier term tends to encourage large values of σ, which is desirable for large regions with coherent motion, except at motion discontinuities.…”