Given a video sequence acquired with a fixed camera, the generation of the stationary background of the scene is a challenging problem which aims at computing a reference image for a motionless background. For that purpose, we developed our method named LaBGen, which emerged as the best one during the Scene Background Modeling and Initialization (SBMI) workshop organized in 2015, and the IEEE Scene Background Modeling Contest (SBMC) organized in 2016. LaBGen combines a pixel-wise temporal median filter and a patch selection mechanism based on motion detection. To detect motion, a background subtraction algorithm decides, for each frame, which pixels belong to the background. In this paper, we describe the LaBGen method extensively, evaluate it on the SBI 2016 dataset and compare its performance with other background generation methods. We also study its computational complexity, the performance sensitivity with respect to its parameters, and the stability of the predicted background image over time with respect to the chosen background subtraction algorithm.We provide an open source C++ implementation at http://www.telecom.ulg.ac.be/labgen.
The estimation of the background image from a video sequence is necessary in some applications. Computing the median for each pixel over time is effective, but it fails when the background is visible for less than half of the time. In this paper, we propose a new method leveraging the segmentation performed by a background subtraction algorithm, which reduces the set of color candidates, for each pixel, before the median is applied. Our method is simple and fully generic as any background subtraction algorithm can be used. While recent background subtraction algorithms are excellent in detecting moving objects, our experiments show that the frame difference algorithm is a technique that compare advantageously to more advanced ones. Finally, we present the background images obtained on the SBI dataset, which appear to be almost perfect.
Abstract. The stationary background generation problem consists in generating a unique image representing the stationary background of a given video sequence. The LaBGen background generation method combines a pixel-wise median filter and a patch selection mechanism based on a motion detection performed by a background subtraction algorithm. In our previous works related to LaBGen, we have shown that, surprisingly, the frame difference algorithm provides the most effective motion detection on average. Compared to other background subtraction algorithms, it detects motion between two frames without relying on additional past frames, and is therefore memoryless. In this paper, we experimentally check whether the memoryless property is truly relevant for LaBGen, and whether the effective motion detection provided by the frame difference is not an isolated case. For this purpose, we introduce LaBGen-OF, a variant of LaBGen leverages memoryless dense optical flow algorithms for motion detection. Our experiments show that using a memoryless motion detector is an adequate choice for our background generation framework, and that LaBGen-OF outperforms LaBGen on the SBMnet dataset. We further provide an open-source C++ implementation of both methods at http://www.telecom.ulg.ac.be/labgen.
Abstract-Estimating the stationary background of a video sequence is useful in many applications like surveillance, segmentation, compression, inpainting, privacy protection, and computational photography. To perform this task, we introduce the LaBGen-P method based on the principles of LaBGen and the conclusions drawn in the corresponding paper. It combines a pixel-wise median filter and a pixel selection mechanism based on a motion detection performed by the frame difference algorithm. By working with pixels instead of patches, as originally done in LaBGen, it avoids some discontinuities between different spatial areas and generates better visual results. In this paper, we describe the LaBGen-P method, study its performance on the sequences of the SBMnet dataset, and compare it to that of LaBGen and other methods on the same dataset. Both algorithms emerged as the best ones during the IEEE Scene Background Modeling Contest (SBMC) organized in 2016. However, as there is not yet a good understanding of the recommended metrics, and due to the small amount of video sequences provided with the corresponding ground truth, we have performed a subjective evaluation. More precisely, 35 human experts were asked to compare background images estimated by LaBGen-P and LaBGen, and select the best one. From these experiments, it turns out that the results of LaBGen-P are preferred for about two thirds of the video sequences. Note that we provide an open-source C++ implementation at http://www.telecom.ulg.ac.be/labgen.
Given a video sequence acquired from a fixed camera, the stationary background generation problem consists of generating a unique image estimating the stationary background of the sequence. During the IEEE Scene Background Modeling Contest (SBMC) organized in 2016, we presented the LaBGen-P method. In short, this method relies on a motion detection algorithm for selecting, for each pixel location, a given amount of pixel intensities that are most likely static by keeping the ones with the smallest quantities of motion. These quantities are estimated by aggregating the motion scores returned by the motion detection algorithm in the spatial neighborhood of the pixel. After this selection process, the background image is then generated by blending the selected intensities with a median filter. In our previous works, we showed that using a temporally-memoryless motion detection, detecting motion between two frames without relying on additional temporal information, leads our method to achieve the best performance. In this work, we go one step further by developing LaBGen-P-Semantic, a variant of LaBGen-P, the motion detection step of which is built on the current frame only by using semantic segmentation. For this purpose, two intra-frame motion detection algorithms, detecting motion from a unique frame, are presented and compared. Our experiments, carried out on the Scene Background Initialization (SBI) and SceneBackgroundModeling.NET (SBMnet) datasets, show that leveraging semantic segmentation improves the robustness against intermittent motions, background motions and very short video sequences, which are among the main challenges in the background generation field. Moreover, our results confirm that using an intra-frame motion detection is an appropriate choice for our method and paves the way for more techniques based on semantic segmentation.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.