Segmentation and 2D motion estimation by region fragments

Abstract

In estimating multiple image motions, a central problem is that 2D motion estimation and region segmentation are mutually dependent. This paper presents a new region description method for dealing with this mutual dependence problem. Segmentation and motion esiimation are simultaneously performed by a clustering process based on color, motion, and pixel position. As a result of the clustering, an image is decomposed into region fmgments. Each fmgment is characterized by distribution parameters of color, pixel positions, and spatiotemporal intensity gradients. The image is described by the parameters of the region fragments. 2D motion vectors for each fragment are obtained from the distribution parameters of the intensity gradients. By combining those features in the clustering process, regions are segmented more precisely, motion boundaries are not blurred, and the 2D motions are obtained even in noisy areas. Experimental results of this new method are shown in this paper.

Motion estimation with quadtree splines

“…These methods can be extended to more complex scenes, however, by using a collection of global motion models. For example, each pixel can be associated with one of several global motion hypotheses, resulting in a layered motion model [Wang and Adelson, 1993;Jepson and Black, 1993;Etoh and Shirai, 1993;Bober and Kittler, 1993]. Alternatively, a single image can be recursively subdivided into smaller parametric motion patches based on estimates of the current residual error in the flow estimate [Müller et al, 1994].…”

Direct incremental model-based image motion segmentation for video analysis

“…The resulting motion estimates, on which the clustering relies, are usually noisy. Moreover, these bottom-up algorithms [7,11,24], suited for the two-frame case, cannot easily incorporate the temporal aspect and the benefit from a predicted segmentation map.…”

“…Moreover, the determination of the support layers greatly suffers from the lack of competition between different motion models to explain the motion measurements at pixel locations. Clustering methods [7,1,11], in a wide sense, fall into the second category. For instance, in [7], an unsupervised clustering technique mixes information on the position, color and motion-based residual at every pixel in a competitive learning scheme, whereas in [1], a k-mean technique is employed to group regions based on their pre-computed affine motion.…”

“…Clustering methods [7,1,11], in a wide sense, fall into the second category. For instance, in [7], an unsupervised clustering technique mixes information on the position, color and motion-based residual at every pixel in a competitive learning scheme, whereas in [1], a k-mean technique is employed to group regions based on their pre-computed affine motion. One important shortcoming of these methods is that clustering in the parameter space is usually sensitive to the number of specified clusters.…”

Tracking non-rigid, moving objects based on color cluster flow

“…This yields another advantage compared to the CBF: No explicit matching of corresponding clusters is required. A similar clustering technique was proposed by [5]. This approach also included spatio-temporal intensity gradients in the clustering procedure.…”