A Deep Learning Approach to Predict Crowd Behavior Based on Emotion

Varghese, Elizabeth; Thampi, Sabu M.

doi:10.1007/978-3-030-04375-9_25

Cited by 22 publications

(15 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many studies in facial affective computing have been published over the past few years [4][5][6] since facial affect plays an important role in human-computer interaction [7][8][9]. For example, facial affect computing is a functional supplement for intelligent surveillance [10] when combining with multiple moving targets tracking technologies [11] to detect the facial affect of the crowd in video surveillance to avoid potential dangers and disasters. Previous works mainly focused on the categorical model based on six basic facial expressions (Happiness, Sadness, Fear, Anger, Surprise and Disgust) defined by Ekman et al [2].…”

Section: Related Workmentioning

confidence: 99%

Real-Time Facial Affective Computing on Mobile Devices

Guo

Xia

Wang

et al. 2020

Sensors

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) have become one of the state-of-the-art methods for various computer vision and pattern recognition tasks including facial affective computing. Although impressive results have been obtained in facial affective computing using CNNs, the computational complexity of CNNs has also increased significantly. This means high performance hardware is typically indispensable. Most existing CNNs are thus not generalizable enough for mobile devices, where the storage, memory and computational power are limited. In this paper, we focus on the design and implementation of CNNs on mobile devices for real-time facial affective computing tasks. We propose a light-weight CNN architecture which well balances the performance and computational complexity. The experimental results show that the proposed architecture achieves high performance while retaining the low computational complexity compared with state-of-the-art methods. We demonstrate the feasibility of a CNN architecture in terms of speed, memory and storage consumption for mobile devices by implementing a real-time facial affective computing application on an actual mobile device.

show abstract

Section: Related Workmentioning

confidence: 99%

Real-Time Facial Affective Computing on Mobile Devices

Guo

Xia

Wang

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…The group size is once more increased with the consideration of two studies gathering their data from surveillance videos. [26] consider outdoor videos, and [27] consider both outdoor and indoor video footage. These videos include people in a natural context, for example walking on the street or exercising.…”

Section: Group Typesmentioning

confidence: 99%

“…They substantiate the omitting of disgust and surprise by referring to [33], in which it is stated that the facial expression of disgust is similar to that of anger, and that the same goes for surprise and fear. In [26] surprise and disgust are also omitted, and excited and neutral added instead. This follows the dataset they use (MED, see Section 4).…”

Section: Emotion Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Emotion Recognition for Groups: A Review

Veltmeijer

Gerritsen

Hindriks

2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

This review aims to summarize and describe research on the topic of automatic group emotion recognition. In recent years, the topic of emotion analysis of groups or crowds has gained interest, with studies performing emotion detection in different contexts, using different datasets and modalities (such as images, video, audio, social media messages), and taking different approaches. Articles are included after an innovative search method, including Dense Query Extraction and automatic cross-referencing. Discussed are the types of groups and emotion models considered in automatic emotion recognition research, common datasets for all modalities, general approaches taken, and reported performances. These performances are discussed, followed by an analysis of the application possibilities of the discussed methods. To ensure clear, replicable, and comparable studies, we suggest research should test on multiple, common datasets and report on multiple metrics, when possible. Implementation details and code should be made available where possible. An area of interest for future work is to build systems with more real-world application possibilities, coping with changing group sizes, different emotional subgroups, and changing emotions over time, while having a higher robustness and working with datasets with reduced biases.

show abstract

“…At the crowd level (for overviews, see Grant andFlynn 2017, Varghese andThampi 2018), researchers have worked at ways to automatically assess, for example, the density of the crowd (Almagbile 2019, Rodriguez, Sivic andLaptev 2017), the level of excitement in the crowd (Baig, Barakova, Marcenaro et al 2014, Conigliaro, Rota, Setti et al 2015, Varghese and Thampi 2018, and the presence of anomalies in the crowd (for overviews, see Nayak, Pati and Das 2021, Sodemann, Ross and Borghetti 2012, Tripathi, Jalal and Agrawal 2018. Removing crowd size limitations of prior methods, Cruz and González-Villa (2021) proposed a method to estimate crowd sizes on high-resolution images from gigapixel cameras, in which images typically contain many thousands of individuals.…”

Section: Recognition Of Persons Groups and Crowdsmentioning

confidence: 99%

Promise Into Practice: Application of Computer Vision in Empirical Research on Social Distancing

Bernasco¹,

Hoeben²,

Koelma³

et al. 2021

Preprint

View full text Add to dashboard Cite

Social scientists increasingly use video data, but large-scale analysis of its content is often constrained by scarce manual coding resources. Upscaling may be possible with the application of automated coding procedures, which are being developed in the field of computer vision. Here, we introduce computer vision to social scientists, review the state-of-the-art in relevant subfields, and provide a working example of how computer vision can be applied in empirical sociological work. Our application involves defining a ground truth by human coders, developing an algorithm for automated coding, testing the performance of the algorithm against the ground truth, and run the algorithm on a large-scale dataset of CCTV images. The working example concerns monitoring social distancing behavior in public space over more than a year of the COVID-19 pandemic. Finally, we discuss prospects for the use of computer vision in empirical social science research and address technical and ethical limitations.

show abstract

A Deep Learning Approach to Predict Crowd Behavior Based on Emotion

Cited by 22 publications

References 21 publications

Real-Time Facial Affective Computing on Mobile Devices

Real-Time Facial Affective Computing on Mobile Devices

Automatic Emotion Recognition for Groups: A Review

Promise Into Practice: Application of Computer Vision in Empirical Research on Social Distancing

Contact Info

Product

Resources

About