SuBSENSE: A Universal Change Detection Method With Local Adaptive Sensitivity

Abstract

Foreground/background segmentation via change detection in video sequences is often used as a stepping stone in high-level analytics and applications. Despite the wide variety of methods that have been proposed for this problem, none has been able to fully address the complex nature of dynamic scenes in real surveillance tasks. In this paper, we present a universal pixel-level segmentation method that relies on spatiotemporal binary features as well as color information to detect changes. This allows camouflaged foreground objects to be detected more easily while most illumination variations are ignored. Besides, instead of using manually set, frame-wide constants to dictate model sensitivity and adaptation speed, we use pixel-level feedback loops to dynamically adjust our method's internal parameters without user intervention. These adjustments are based on the continuous monitoring of model fidelity and local segmentation noise levels. This new approach enables us to outperform all 32 previously tested state-of-the-art methods on the 2012 and 2014 versions of the ChangeDetection.net dataset in terms of overall F-Measure. The use of local binary image descriptors for pixel-level modeling also facilitates high-speed parallel implementations: our own version, which used no low-level or architecture-specific instruction, reached real-time processing speed on a midlevel desktop CPU. A complete C++ implementation based on OpenCV is available online.

Deep background subtraction with scene-specific convolutional neural networks

“…Fig. 4 shows segmentation results of our algorithm as well as for other traditional (GMM [1] and KDE [2]), and state-of-the-art (IUTIS-5 [18] and SuB-SENSE [7]) methods. Table II shows that the quality of our ConvNet based background subtraction algorithm is similar to that of stateof-the-art methods when the training data are generated with IUTIS-5 [18].…”

“…Typical segmentation results for several sequences of CDnet 2014 [13]. Columns from left to right show the input image, the ground truth and the segmentation masks of ConvNet-GT, ConvNet-IUTIS, IUTIS-5 [18], SuBSENSE [7], GMM [1] and KDE [2].…”

“…authors have worked on other components such as postprocessing operations [6] or feedback loops to update model parameters ( [3], [7]). In contrast, the subtraction operation is rarely explored.…”

“…In contrast, the subtraction operation is rarely explored. It often consists in a simple probability thresholding operation at the pixel level ( [1], [2]) or in a matching mechanism with collected samples ( [7], [8]). In addition, it is almost exclusively based on low-level features such as color components ( [1], [8]), gradients [9], or handcrafted features such as the LBP histogram [10], LBSP [7] and matrix covariance descriptor [11].…”

A Perfect Estimation of a Background Image Does Not Lead to a Perfect Background Subtraction: Analysis of the Upper Bound on the Performance

“…Note that even if other types of features are sometimes considered (e.g. local binary patterns in [9], local binary similarity patterns in [13], or gradients [8]), we limit the scope of this paper to colors.…”