A 3D Atrous Convolutional Long Short-Term Memory Network for Background Subtraction

Hu, Zhihang; Turki, Turki; Phan, NhatHai; Wang, Jianli

doi:10.1109/access.2018.2861223

Cited by 53 publications

(38 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But, it may not be the case in certain applications as pointed out by Hu et al [82]. ; 3) Each pixel is processed independently and then the foreground mask may contain isolated false positives and false negatives; 4) It is computationally expensive due to large number of patches extracted from each frame as remarked by Lim and Keles [114]; 5) it requires preprocessing or post-processing of the data, and hence is not based on an end-to-end learning framework [82]; 6) ConvNet use few frames as input and thus can not consider long-term dependencies of the input video sequences [82]; and 7) ConvNet is a deep encoder-decoder network that is a generator network. But, the classical generator networks produce blurry foreground regions and such networks can not preserve the objects edges because they minimize the classical loss functions (e.g., Euclidean distance) between the predicted output and the ground-truth [117].…”

Section: Convolutional Neural Networkmentioning

confidence: 99%

“…In another work, Hu et al [82] developed a 3D atrous CNN model to learn deep spatial-temporal features without losing resolution information. In addition, this model is combined with two convolutional long short-term memory (ConvLSTM) networks in order to capture both short-term and long-term spatio-temporal information of the input video data.…”

Section: D-cnnsmentioning

confidence: 99%

“…We compared the F-measures obtained by the different algorithms with the F-measures of other representative background subtraction algorithms over the complete evaluation dataset: (A) two conventional statistical models (MOG [180], RMOG [192], (B) three advanced non-parametric models (SubSENSE [178], PAWCS [179], and Spectral-360 [165]), and (C) two conventional neural networks models (SOBS-CF [125], SC-SOBS [126] [161], 3D Atrous CNN [82]). -Generative Adversarial Networks: We grouped scores of 4 GAN algorithms:…”

Section: Quantitative Evaluationmentioning

confidence: 99%

See 2 more Smart Citations

Deep neural network concepts for background subtraction:A systematic review and comparative evaluation

et al. 2019

View full text Add to dashboard Cite

Conventional neural networks show a powerful framework for background subtraction in video acquired by static cameras. Indeed, the well-known SOBS method and its variants based on neural networks were the leader methods on the largescale CDnet 2012 dataset during a long time. Recently, convolutional neural networks which belong to deep learning methods were employed with success for background initialization, foreground detection and deep learned features. Currently, the top current background subtraction methods in CDnet 2014 are based on deep neural networks with a large gap of performance in comparison on the conventional unsupervised approaches based on multi-features or multi-cues strategies. Furthermore, a huge amount of papers was published since 2016 when Braham and Van Droogenbroeck published their first work on CNN applied to background subtraction providing a regular gain of performance. In this context, we provide the first review of deep neural network concepts in background subtraction for novices and experts in order to analyze this success and to provide further directions. For this, we first surveyed the methods used background initialization, background subtraction and deep learned features. Then, we discuss the adequacy of deep neural networks for background subtraction. Finally, experimental results are presented on the CDnet 2014 dataset.

show abstract

Section: Convolutional Neural Networkmentioning

confidence: 99%

Section: D-cnnsmentioning

confidence: 99%

Section: Quantitative Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Deep neural network concepts for background subtraction:A systematic review and comparative evaluation

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Similar decomposition schemes have been used by Denton and Birodkar [18] and Villegas et al [19] who developed the MCnet+Res system. These decomposition methods borrow the idea from background subtraction techniques [20] and work well in rather simple scenarios. Hou et al [10] introduced a bidirectional constraint network, BCnet-D, and used a video adversarial loss function to constrain the motion of predictions.…”

Section: Related Workmentioning

confidence: 99%

Generative Adversarial Networks for Stochastic Video Prediction With Action Control

Turki

Wang

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

The ability of predicting future frames in video sequences, known as video prediction, is an appealing yet challenging task in computer vision. This task requires an in-depth representation of video sequences and a deep understanding of real-word causal rules. Existing approaches for tackling the video prediction problem can be classified into two categories: deterministic and stochastic methods. Deterministic methods lack the ability of generating possible future frames and often yield blurry predictions. On the other hand, although current stochastic approaches can predict possible future frames, their models lack the ability of action control in the sense that they cannot generate the desired future frames conditioned on a specific action. In this paper, we propose new generative adversarial networks (GANs) for stochastic video prediction. Our framework, called VPGAN, employs an adversarial inference model and a cycle-consistency loss function to empower the framework to obtain more accurate predictions. In addition, we incorporate a conformal mapping network structure into VPGAN to enable action control for generating desirable future frames. In this way, VPGAN is able to produce fake videos of an object moving along a specific direction. Experimental results show that the combination of VPGAN with a pre-trained image segmentation model outperforms existing stochastic video prediction methods. INDEX TERMS Cycle consistency, deep learning, generative adversarial networks, video prediction.

show abstract

“…SFEN [21] first extracts semantic maps from a single frame as the input of a ConvLSTM, and an STN model [22] and CRF [23] are combined to enhance the motion robustness and spatial smoothness of the output mask. Hu et al [24] conduct 3D atrous convolutional network with multi-frame input before ConvLSTM. By contrast to the sequence-based method, the foreground segmentation methods [11], [12] only consider a single frame to segment foreground.…”

Section: Related Workmentioning

confidence: 99%

Universal Foreground Segmentation Based on Deep Feature Fusion Network for Multi-Scene Videos

2019

View full text Add to dashboard Cite

Foreground/background (fg/bg) classification is an important first step for several video analysis tasks such as people counting, activity recognition and anomaly detection. As is the case for several other Computer Vision problems, the advent of deep Convolutional Neural Network (CNN) methods has led to major improvements in this field. However, despite their success, CNN-based methods have difficulties in coping with multi-scene videos where the scenes change multiple times along the time sequence. In this paper, we propose a deep features fusion network based foreground segmentation method (DFFnetSeg), which is both robust to scene changes and unseen scenes comparing with competitive state-of-the-art methods. In the heart of DFFnetSeg lies a fusion network that takes as input deep features extracted from a current frame, a previous frame, and a reference frame and produces as output a segmentation mask into background and foreground objects. We show the advantages of using a fusion network and the three frames group in dealing with the unseen scene and bootstrap challenge. In addition, we show that a simple reference frame updating strategy enables DFFnetSeg to be robust to sudden scene changes inside video sequences and prepare a motion map based post-processing method which further reduces false positives. Experimental results on the test dataset generated from CDnet2014 and Lasiesta demonstrate the advantages of the DFFnetSeg method. INDEX TERMS Convolutional neural network, foreground segmentation, multi-scene videos aware.

show abstract

A 3D Atrous Convolutional Long Short-Term Memory Network for Background Subtraction

Cited by 53 publications

References 28 publications

Deep neural network concepts for background subtraction:A systematic review and comparative evaluation

Deep neural network concepts for background subtraction:A systematic review and comparative evaluation

Generative Adversarial Networks for Stochastic Video Prediction With Action Control

Universal Foreground Segmentation Based on Deep Feature Fusion Network for Multi-Scene Videos

Contact Info

Product

Resources

About