Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories

Abstract

In this paper, we study the problem of segmenting tracked feature point trajectories of multiple moving objects in an image sequence. Using the affine camera model, this problem can be cast as the problem of segmenting samples drawn from multiple linear subspaces. In practice, due to limitations of the tracker, occlusions, and the presence of nonrigid objects in the scene, the obtained motion trajectories may contain grossly mistracked features, missing entries, or corrupted entries. In this paper, we develop a robust subspace separation scheme that deals with these practical issues in a unified mathematical framework. Our methods draw strong connections between lossy compression, rank minimization, and sparse representation. We test our methods extensively on the Hopkins155 motion segmentation database and other motion sequences with outliers and missing data. We compare the performance of our methods to state-of-the-art motion segmentation methods based on expectation-maximization and spectral clustering. For data without outliers or missing information, the results of our methods are on par with the state-of-the-art results and, in many cases, exceed them. In addition, our methods give surprisingly good performance in the presence of the three types of pathological trajectories mentioned above. All code and results are publicly available at http://perception.csl.uiuc.edu/coding/motion/.

A closed form solution to robust subspace estimation and clustering

“…While in principle this problem appears to be very similar to those in (15) and (18), there are a number of key differences.…”

“…In the case of outliers, following [15], it is assumed that the outliers are sparse. Since both C and E are sparse, the equation…”

“…|E jk | 2 is the 2.1 norm of the matrix of errors E. Notice that this problem is analogous to (15) and (16), except that the 1 and the Frobenious norms are replaced by the nuclear and the 2,1 norms, respectively. It is argued in [13] that this allows one to better handle outliers in one data point but not in others, since it is a convex relaxation to the number of corrupted data points.…”

“…We then assume that the data are obtained by adding noise to the clean dictionary, i.e., D = A + E. As a consequence, our method searches simultaneously for a clean dictionary, the sparse coefficients and the noise. Second, the main difference with (15) is that the 1 norm of the matrix of coefficients is replaced by the nuclear norm. Third, the main difference with (18) is that the 2,1 norm of the matrix of noise is replaced by the Frobenius norm.…”

“…where, following [15,7,2], the 1 penalty on the matrix of outliers is motivated by the assumption that the outliers are sparse. As in the case of noise, the major difference of this formulation with respect to (16) and (18) is that, rather than using a corrupted dictionary, we search simultaneously for a clean dictionary A, the sparse coefficients C and the outliers E. Also, notice that the 1 norm of the matrix of coefficients is replaced by the nuclear norm and that the 2,1 norm of the matrix of errors is replaced by the 1 norm.…”