A Hybrid Model for Driver Emotion Detection Using Feature Fusion Approach

Sukhavasi, Suparshya Babu; Sukhavasi, Susrutha Babu; Elleithy, Khaled; El-Sayed, Ahmed; Elleithy, Abdelrahman

doi:10.3390/ijerph19053085

Cited by 19 publications

(10 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are also situations in which the proposed methods exceed the state-of-the-art performances [ 38 , 39 , 54 , 62 , 74 , 81 , 101 ]. Similarly, context-aware solutions for emotion recognition [ 47 , 49 , 50 , 98 ] or practical solutions [ 37 , 124 , 127 , 130 , 133 , 159 ] demonstrate promising results.…”

Section: Discussionmentioning

confidence: 99%

New Trends in Emotion Recognition Using Image Analysis by Neural Networks, a Systematic Review

Cîrneanu,

Popescu,

Iordache

2023

Sensors

View full text Add to dashboard Cite

Facial emotion recognition (FER) is a computer vision process aimed at detecting and classifying human emotional expressions. FER systems are currently used in a vast range of applications from areas such as education, healthcare, or public safety; therefore, detection and recognition accuracies are very important. Similar to any computer vision task based on image analyses, FER solutions are also suitable for integration with artificial intelligence solutions represented by different neural network varieties, especially deep neural networks that have shown great potential in the last years due to their feature extraction capabilities and computational efficiency over large datasets. In this context, this paper reviews the latest developments in the FER area, with a focus on recent neural network models that implement specific facial image analysis algorithms to detect and recognize facial emotions. This paper’s scope is to present from historical and conceptual perspectives the evolution of the neural network architectures that proved significant results in the FER area. This paper endorses convolutional neural network (CNN)-based architectures against other neural network architectures, such as recurrent neural networks or generative adversarial networks, highlighting the key elements and performance of each architecture, and the advantages and limitations of the proposed models in the analyzed papers. Additionally, this paper presents the available datasets that are currently used for emotion recognition from facial expressions and micro-expressions. The usage of FER systems is also highlighted in various domains such as healthcare, education, security, or social IoT. Finally, open issues and future possible developments in the FER area are identified.

show abstract

Section: Discussionmentioning

confidence: 99%

New Trends in Emotion Recognition Using Image Analysis by Neural Networks, a Systematic Review

Cîrneanu,

Popescu,

Iordache

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…There are two main detection tasks for emotion detection: discrete emotion detection (classification task for basic emotions) and dimensional emotion detection (regression task for valence, arousal and dominance). As shown in Table 5, most of the facialexpression-based studies choose the discrete emotion (six basic emotions and neutral) as the detection targets, and the deep neural network, such as CNN-based models, are commonly used in facial classification, such as GLFCNN [82], Xception [86,87], and basic CNN [85,90]. Equations ( 1) and (2) show the most commonly used loss function (multiclass cross-entropy loss and binary cross-entropy loss) for facial-expression-based emotion detection [82,85,86,88].…”

Section: Facial-expression-basedmentioning

confidence: 99%

Review and Perspectives on Human Emotion for Connected Automated Vehicles

Li,

Tan

et al. 2024

Automot. Innov.

View full text Add to dashboard Cite

The progression toward automated driving and the latest advancement in vehicular networking have led to novel and natural human-vehicle-road systems, in which affective human-vehicle interaction is a crucial factor affecting the acceptance, safety, comfort, and traffic efficiency of connected and automated vehicles (CAVs). This development has inspired increasing interest in how to develop affective interaction framework for intelligent cockpit in CAVs. To enable affective human-vehicle interactions in CAVs, knowledge from multiple research areas is needed, including automotive engineering, transportation engineering, human–machine interaction, computer science, communication, as well as industrial engineering. However, there is currently no systematic survey considering the close relationship between human-vehicle-road and human emotion in the human-vehicle-road coupling process in the CAV context. To facilitate progress in this area, this paper provides a comprehensive literature survey on emotion-related studies from multi-aspects for better design of affective interaction in intelligent cockpit for CAVs. This paper discusses the multimodal expression of human emotions, investigates the human emotion experiment in driving, and particularly emphasizes previous knowledge on human emotion detection, regulation, as well as their applications in CAVs. The promising research perspectives are outlined for researchers and engineers from different research areas to develop CAVs with better acceptance, safety, comfort, and enjoyment for users.

show abstract

“…Xiao et al [4] designed a driver facial expression recognition model where the deep convolutional network was pretrained a on FER and CK+ data and then fine-tuned on the collected driver data. Sukhavasi et al [38] proposed a hybrid driver expression recognition model by fusing deep neural networks with support vector machine.…”

Section: Driver Facial Expression Recognitionmentioning

confidence: 99%

“…Sukhavasi et al. [38] proposed a hybrid driver expression recognition model by fusing deep neural networks with support vector machine.…”

Section: Literature Reviewmentioning

confidence: 99%

Transferable driver facial expression recognition based on joint discriminative correlation alignment network with enhanced feature attention

Chen,

Du,

Deng

et al. 2023

IET Intelligent Trans Sys

View full text Add to dashboard Cite

Facial expression is closely related to the emotions of drivers, thus facilitating safe driving detection in advanced driving assistance system (ADAS). Recently, deep learning techniques have become prevalent for facial expression recognition. However, for driving scenarios, the facial expression recognition is mainly challenged by the problems of small sample size as well as effective feature representation. To address the above issues, this paper puts forward a transfer learning‐based method for driver facial expression recognition by fully exploiting the other sources of facial expression data that may obey different distributions. Specifically, an enhanced feature attention module is firstly devised such that affluent features with multi‐scales can be extracted and refined based on the spatial and channel attention mechanisms. Then, a joint correlation alignment loss is presented ensuring that the samples in the source and target domains are transformed into the shared common subspace to reduce difference of both the marginal and conditional distributions. Multiple transfer learning tasks on real‐world data are carried out to evaluate the proposed method. The experimental results show that our model achieves better recognition accuracy for driver facial expression compared with several traditional and deep learning‐based transfer learning algorithms.

show abstract

A Hybrid Model for Driver Emotion Detection Using Feature Fusion Approach

Cited by 19 publications

References 53 publications

New Trends in Emotion Recognition Using Image Analysis by Neural Networks, a Systematic Review

New Trends in Emotion Recognition Using Image Analysis by Neural Networks, a Systematic Review

Review and Perspectives on Human Emotion for Connected Automated Vehicles

Transferable driver facial expression recognition based on joint discriminative correlation alignment network with enhanced feature attention

Contact Info

Product

Resources

About