Spatio-temporal segmentation based on region merging

Abstract

This paper proposes a technique for spatiotemporal segmentation to identify the objects present in the scene represented in a video sequence. This technique processes two consecutive frames at a time. A region-merging approach is used to identify the objects in the scene. Starting from an oversegmentation of the current frame, the objects are formed by iteratively merging regions together. Regions are merged based on their mutual spatiotemporal similarity. The spatiotemporal similarity measure takes both temporal and spatial information into account, the emphasis being on the former. We propose a Modified Kolmogorov-Smirnov test for estimating the temporal similarity. This test efficiently uses temporal information in both the residual distribution and the motion parametric representation. The region-merging process is based on a weighted, directed graph. Two complementary graph-based clustering rules are proposed, namely, the strong rule and the weak rule. These rules take advantage of the natural structures present in the graph. Also, the rules take into account the possible errors and uncertainties reported in the graph. The weak rule is applied after the strong rule. Each rule is applied iteratively, and the graph is updated after each iteration. Experimental results on different types of scenes demonstrate the ability of the proposed technique to automatically partition the scene into its constituent objects.

EdgeFlow: a technique for boundary detection and image segmentation

“…[1], [15], [26], [27], global optimization techniques based on energy functions and Bayesian criteria [12], [13], and texture segmentation [2], [9]- [11], [16], [19], [22], [24], [31]. In [39], a region competition approach has been proposed to unify the active contour model, region growing, and Bayes for image segmentation.…”

Automatic detection of salient objects and spatial relations in videos for a video database system

“…The details of moving object segmentation and spatiotemporal segmentation can be found in [14,[20][21][22][23][24][25][26] and tracking in [27][28][29][30][31].…”

Into the woods: visual surveillance of noncooperative and camouflaged targets in complex outdoor settings

“…Moscheni, et.al. [39] developed techniques for video coding and robot vision that work only on two consecutive frames. Both spatial and temporal information is used to compute a similarity between regions.…”