Pixel matching and motion segmentation in image sequences

Abstract

Abstract. This paper presents a coarse-to-fine algorithm to obtain pinel trajectories in a long image sequence and to segment it into subsets corresponding to distinctly moving objects. Much of the previous related work has addressed the computation of optical flow over two frames or sparse feature trajectories in sequences. The features used are often small in number and restrictive assumptions are made about them such as the visibility of features in all the frames. The algorithm described here uses a coarse scale point feature detector to form a 3-D dot pattern in the spatiotemporal space. The trajectories are extracted as 3-D curves formed by the points using perceptual grouping. Increasingly dense correspondences are obtained iteratively from the sparse feature trajectories. At the finest level, which is the focus of this paper, all pixels are matched and the finest boundaries of the moving objects are obtained. Keywords: Motion Segmentation, Perceptual Grouping, Pixel Matching, Triangulation, Feature Matching, Optical Flow. IntroductionThis paper describes a component of our work aimed at interpretation of image sequences. Given an image sequence containing an arbitrary number of rigid objects in motion, the objectives of the overall work are to identify feature points in the scene, obtain spatially dense trajectories of those points, segment moving objects, compute image flow at each pixel, and derive a qualitative description of the scene structure and dynamics from the image sequence. Such qualitative interpretation of the image sequence is useful for a variety of applications such as traffic scene analysis, biological image analysis and aerial image understanding. The focus of this paper is on the detection of pixel flow trajectories. Next section reviews some related previous work. Section 3 summarizes the steps of coarseto-fine detection of sparse feature trajectories. Section 4 then gives the details of the algorithm for finding pixel flow trajectories which is the objective of this paper. Section 5 presents experimental results and Section 6 presents concluding remarks.

Estimating 3D object parameters from 2D grey-level images

“…Figure 1.1 shows an example of an image of a scene consisting of simple objects. The image is produced by PARSCENE 1 . The image has a resolution of 256x256 pixels and grey-levels between zero and 255.…”

“…In 2.3.1 several co-ordinate systems are introduced to make the modelling transparent and modular. In 2.3.2 the geometric model of the body, the camera and the light 1 Physical laws to describe dynamics, as is already being done in virtual reality (see e.g. http://ligwww.epfl.ch/), are not implemented.…”

“…As was mentioned before, the velocity vector of a pixel is called an optic flow vector, and estimating it for each pixel in an image results in an optic flow image ( [23], p. 598). Optic flow can be computed with a feature-based [1], region-based or gradient-based method [21], [34].…”

“…Image motion can be used to segment images into different regions. When the motion on image level is estimated using either a featurebased, region-based or a gradient-based method, it can be used to segment images into different depth-layers [26] or regions corresponding with different objects and background [1]. This type of segmentation is called motion-based segmentation.…”

“…The eigenvalues are non-negative because the matrix can be written as a product H T H, which is always positive semi-definite 1 . From this property it follows that y T H T Hy is always ≥ 0 for ∀y∈R p , with p the number of parameters.…”