A Spatio-temporal Filtering Approach to Motion Segmentation

Abstract

Abstract. In this paper, a new frequency-based approach to motion segmentation is presented. The proposed technique represents the sequence as a spatio-temporal volume, where a moving object corresponds to a three-dimensional object. In order to detect the "3D volumes" corresponding to significant motions, a new scheme based on a band-pass filtering with a set of logGabor spatio-temporal filters is used. It is well known that one of the main problems of these approaches is that a filter response varies with the spatial orientation of the underlying signal. To solve this spatial dependency, the proposed model allows to recombine information of motions that has been separated in several filter responses due to its spatial structure. For this purpose, motions are detected as invariance in statistical structure across a range of spatio-temporal frequency bands. This technique is illustrated on real and simulated data sets, including sequences with occlusion and transparencies.

A New Approach to Motion Pattern Recognition and Its Application to Optical Flow Estimation

“…In [22], we introduce the use of local energy peaks for selecting "relevant points," but in that case the number of peaks is very high and the execution time increases significantly. To extract less peaks (just the most significant ones), we introduce in this paper a selection based on a hysteresis procedure [39].…”

“…The combination of any subset of separable features will define an integral feature at a given point (x, y, z). In this paper, the following five separable features proposed in [22] and [40] will be used: phase, local energy, local standard deviation, local contrast of the local energy, and local entropy. In addition, three new separable features, more related with apparent motion information, have been tested: the first and second derivatives at each point, and the estimation of the apparent motion given by (23).…”

“…where w r is a weighting function that gives more influence to elements at the center of the neighborhood than to those at the periphery, with the points in the neighborhood indexed by r, and where M r = M(x r , y r , t) and d r = d(x r , y r , t) are defined as in (22).…”

“…This new scheme recombines responses that, even with different spatial characteristics, have continuity in its motion. Preliminary results were obtained in [22] for translation motions; in this paper, we improve the different stages of the model, allowing us to face more complex motions like rotations or articulated object movements.…”

Dealing with Multiple Motions in Optical Flow Estimation

“…In order to confront this problem, in this paper we develop a methodology for optical flow estimation that is able to represent multiple velocities at the same point. To detect points with multiple motions, the model introduced in [8] is used. This model is a frequency-based approach that groups spatio-temporal filter responses with continuity in its motion (each group will define a motion pattern).…”