Illumination-Based Data Augmentation for Robust Background Subtraction

Abstract

A core challenge in background subtraction (BGS) is handling videos with sudden illumination changes in consecutive frames. In this paper, we tackle the problem from a data point-of-view using data augmentation. Our method performs data augmentation that not only creates endless data on the fly, but also features semantic transformations of illumination which enhance the generalisation of the model. It successfully simulates flashes and shadows by applying the Euclidean distance transform over a binary mask generated randomly. Such data allows us to effectively train an illumination-invariant deep learning model for BGS. Experimental results demonstrate the contribution of the synthetics in the ability of the models to perform BGS even when significant illumination changes take place. The source code of the project is made publicly available at https://github.com/dksakkos/illumination augmentation.

Image editing-based data augmentation for illumination-insensitive background subtraction

“…Finally, when the proposed augmentation methods are combined with the default ones, the model accuracy reaches an excellent 80.6%. That is an improvement of 4.36% FM score than GL proposed in our pilot study [12]. Therefore, we can deduce that each new method introduces modifications to the training data that offer improvements in different areas, and also all methods complement each other.…”

“…The GL model, after applying the post-processing method 0. GL plus all the above plus default augmentation 0.2 Table 3: Comparison between no augmentation, common augmentation and method proposed in our pilot study [12] which covers global and local illumination changes.…”

“…Such augmented data allows us to provide extra semantic information to the BGS model in terms of illumination for better generalisation performance. The pilot study, published in [12], demonstrated the effectiveness and feasibility of such an approach. In this paper, we extend the work by introducing new data augmentation techniques to handle additional variations to the input image locally and globally.…”

Random Shadows and Highlights: A new data augmentation method for extreme lighting conditions

“…It performs occupancy estimation instead of classification, by mixing the four class labels with ratios proportional to the areas of the cropped images. The method proposed in [15] is closest to our proposed strategy. Nevertheless, they only create illumination circles on the images which does not represent the real world lighting perturbations often seen in indoor environments or even outdoors caused by shadows of the buildings.…”

“…While in Color Jitter, we randomly change the brightness, contrast, saturation and hue of the image. Moreover, we also compare our results with the method proposed in [15], we call it Disk Illumination. To study over-fitting, we quantify the Train-Test Difference (TTD), as the name suggests, difference in error on the train set and test set, with and without RSH perturbations.…”