Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects

Sánchez, Francisco Luque; Hupont, Isabelle; Tabik, Siham; Herrera, Francisco

doi:10.1016/j.inffus.2020.07.008

Cited by 59 publications

(27 citation statements)

References 90 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors show the gradual improvement in CNN-based methods such as R-CNN, Fast R-CNN, Deconv-R-CNN, Improved Faster R-CNN, etc., along with other deep learning approaches. Sánchez et al [16] demonstrate a study on crowd behavior analysis or crowd anomaly detection by video processing using deep learning techniques. Authors listed convolutional RBM, Fast R-CNN, 3D CNN, PCANet, deep Gaussian Mixture Model, Convolutional AutoEncoder (CAE) with LSTM (CAE -LSTM), Spatio-temporal CNN, and GAN based approach.…”

Section: A Literature Surveymentioning

confidence: 99%

“…We observed that a missing part is a combined review of various up-to-date video processing functionalities using deep learning techniques in related surveys. The surveys presented above focus only on specific functionality like video anomaly detection [5], abnormal human activity recognition [6], multi-object tracking [10], behavior analysis [16]. None of the surveys collate the research done on various functionality in one survey.…”

Section: B Motivationmentioning

confidence: 99%

See 1 more Smart Citation

Video Processing Using Deep Learning Techniques: A Systematic Literature Review

et al. 2021

View full text Add to dashboard Cite

Studies show lots of advanced research on various data types such as image, speech, and text using deep learning techniques, but nowadays, research on video processing is also an emerging field of computer vision. Several surveys are present on video processing using computer vision deep learning techniques, targeting specific functionality such as anomaly detection, crowd analysis, activity monitoring, etc. However, a combined study is still unexplored. This paper aims to present a Systematic Literature Review (SLR) on video processing using deep learning to investigate the applications, functionalities, techniques, datasets, issues, and challenges by formulating the relevant research questions (RQs). This systematic mapping includes 93 research articles from reputed databases published between 2011 and 2020. We categorize the deep learning technique for video processing as CNN, DNN, and RNN based. We observe the significant advancements in video processing between 2017 and 2020, primarily due to the advent of AlexNet, ResNet, and LSTM based deep learning techniques. The prominent fields of video processing research are observed as human action recognition, crowd anomaly detection, and behavior analysis. This SLR is a helpful guide for the researchers to explore the recent literature, available datasets, and existing deep learning techniques for video processing.

show abstract

Section: A Literature Surveymentioning

confidence: 99%

Section: B Motivationmentioning

confidence: 99%

Video Processing Using Deep Learning Techniques: A Systematic Literature Review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Next, the two features are sent into the pedestrian detection network, respectively, and are fused through a full connection layer in the pedestrian detection network. On the basis of this fusion feature, the pedestrian detection and the prediction of the boundary box of the traveler are carried out by using the YOLO target algorithm through regression [28][29][30]. is pedestrian detection method is the third chapter of the pedestrian detection method based on feature fusion.…”

Section: Design Of Pedestrian Movement Path Detection Systemmentioning

confidence: 99%

Pedestrian Motion Path Detection Method Based on Deep Learning and Foreground Detection

Xie

2021

Complexity

View full text Add to dashboard Cite

For the surveillance video images captured by monocular camera, this paper proposes a method combining foreground detection and deep learning to detect moving pedestrians, making full use of the invariable background of video image. Firstly, the motion region is extracted by the method of interframe difference and background difference. Then, the normalized motion region extracts the feature vectors based on the improved YOLOv3 tiny network. Finally, the trained linear support vector machine is used for pedestrian detection, and the performance of the fusion detection algorithm on caviar dataset is given, which proves the effectiveness of the proposed fusion detection algorithm. Experimental results show that the proposed method not only improves the practical application of pedestrian rerecognition but also reduces the detection range, computational complexity, and false detection rate compared with sliding window method.

show abstract

“…Other works centered around student emotional state detection analyze and process signals from Electroencephalogram (EEG), Electromyogram (EMG), Electrocardiography (ECG), Electrodermal activity (EDA), heart rate variability, skin temperature, blood volume pulse, respiration, or Electrodermography (EDG)/galvanic skin response (GSR) [ 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 ]. Researchers [ 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 ] report the use of deep learning and machine learning (ML) techniques for emotion classification. Finally, other techniques rely on emotion recognition via computer vision [ 22 , 41 , 48 , 49 , 50 ], linguistic semantic approaches [ 51 ], and biological features [ 52 ].…”

Section: Introductionmentioning

confidence: 99%

Wearables for Engagement Detection in Learning Environments: A Review

et al. 2022

View full text Add to dashboard Cite

Appropriate teaching–learning strategies lead to student engagement during learning activities. Scientific progress and modern technology have made it possible to measure engagement in educational settings by reading and analyzing student physiological signals through sensors attached to wearables. This work is a review of current student engagement detection initiatives in the educational domain. The review highlights existing commercial and non-commercial wearables for student engagement monitoring and identifies key physiological signals involved in engagement detection. Our findings reveal that common physiological signals used to measure student engagement include heart rate, skin temperature, respiratory rate, oxygen saturation, blood pressure, and electrocardiogram (ECG) data. Similarly, stress and surprise are key features of student engagement.

show abstract

Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects

Cited by 59 publications

References 90 publications

Video Processing Using Deep Learning Techniques: A Systematic Literature Review

Video Processing Using Deep Learning Techniques: A Systematic Literature Review

Pedestrian Motion Path Detection Method Based on Deep Learning and Foreground Detection

Wearables for Engagement Detection in Learning Environments: A Review

Contact Info

Product

Resources

About