Novel deep learning model for facial expression recognition based on maximum boosted CNN and LSTM

Rajan, Saranya; Chenniappan, P. K.; Devaraj, Somasundaram; Nirmala, M.

doi:10.1049/iet-ipr.2019.1188

Cited by 41 publications

(26 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FER also has a wide variety of social life uses, such as smart protection, lie detection, and smart medical practice [46], [47]. [27] reviewed facial expression recognition models based on deep learning techniques, including DBN, deep CNN, Long Short Term Memory (LSTM) [48,49] and their combination.…”

Section: A Facial Expression Recognitionmentioning

confidence: 99%

Multimodal Emotion Recognition using Deep Learning

Abdullah

Ameen

Sadeeq

et al. 2021

JASTT

153

View full text Add to dashboard Cite

New research into human-computer interaction seeks to consider the consumer's emotional status to provide a seamless human-computer interface. This would make it possible for people to survive and be used in widespread fields, including education and medicine. Multiple techniques can be defined through human feelings, including expressions, facial images, physiological signs, and neuroimaging strategies. This paper presents a review of emotional recognition of multimodal signals using deep learning and comparing their applications based on current studies. Multimodal affective computing systems are studied alongside unimodal solutions as they offer higher accuracy of classification. Accuracy varies according to the number of emotions observed, features extracted, classification system and database consistency. Numerous theories on the methodology of emotional detection and recent emotional science address the following topics. This would encourage studies to understand better physiological signals of the current state of the science and its emotional awareness problems.

show abstract

Section: A Facial Expression Recognitionmentioning

confidence: 99%

Multimodal Emotion Recognition using Deep Learning

Abdullah

Ameen

Sadeeq

et al. 2021

JASTT

153

View full text Add to dashboard Cite

show abstract

“…Facial Expression recognition has been a popular task, one which is also benefiting from the use of an LSTM RNN. This paper [29] feeds a dual CNN structure into an LSTM RNN gate, which can be seen in Figure 4, to process the extracted features from the video frame. These researchers choose to use four different datasets to train and test their model.…”

Section: Model For Facial Expression Recognition Using Lstm Rnnmentioning

confidence: 99%

“…The six basic emotions present in each of these datasets are fear, disgust, anger, happiness, sadness, surprise, and neutral. With their proposed method [29], they were able to attain 99% on CK + dataset, 81.60% on MMI, 56.68% on SFEW (which is highly accurate for that dataset), and 95.21% on their own dataset. Other similar methodologies [30,31] were also able to benefit from the LSTM gate implemented in their models and were evaluated against the MMI dataset.…”

Section: Model For Facial Expression Recognition Using Lstm Rnnmentioning

confidence: 99%

“…Novel Smartphone Authentication Techniques [19] Mouse and Keyboard Based Authentication Methods [23] Handwritten Authentication Methods [27] Model for Facial Expression Recognition Using LSTM RNN [29] Multimodal Expression Recognition Implementing an RNN Approach [32] Motion History Image Expression Recognition [34] Using an RNN to authenticate users through inertial gait recognition or identify users based on their physical movement patterns. Gait recognition also requires gyroscope and accelerameter sensor data to track movement, Authenticate uses a CNN+RNN fusion to detect behavioral patterns in mouse movement.…”

Section: Pros and Consmentioning

confidence: 99%

See 1 more Smart Citation

Applications of Recurrent Neural Network for Biometric Authentication & Anomaly Detection

2021

View full text Add to dashboard Cite

Recurrent Neural Networks are powerful machine learning frameworks that allow for data to be saved and referenced in a temporal sequence. This opens many new possibilities in fields such as handwriting analysis and speech recognition. This paper seeks to explore current research being conducted on RNNs in four very important areas, being biometric authentication, expression recognition, anomaly detection, and applications to aircraft. This paper reviews the methodologies, purpose, results, and the benefits and drawbacks of each proposed method below. These various methodologies all focus on how they can leverage distinct RNN architectures such as the popular Long Short-Term Memory (LSTM) RNN or a Deep-Residual RNN. This paper also examines which frameworks work best in certain situations, and the advantages and disadvantages of each proposed model.

show abstract

“…In order to meet the processing requirements of different length videos, a segmentation strategy is used to build a behavior recognition framework, and a LSTM network model is built based on the spatiotemporal attention of dualstream features, which are used for human behavior recognition in videos [26]. In [27], a novel deep learning framework is proposed, which combines CNN with LSTM cell for real-time facial expression recognition (FER). In [28], a novel deep learning model called LCED which consists of one LSTM-based encoder, features image presentation, and one CNN-based decoder is proposed to weaken the accuracy differences among individuals on activity recognition.…”

Section: Introductionmentioning

confidence: 99%

Research on Safe Driving Evaluation Method Based on Machine Vision and Long Short-Term Memory Network

Shi

Tang²

2021

Journal of Electrical and Computer Engineering

View full text Add to dashboard Cite

The rapid development of transportation industry has brought some potential safety hazards. Aiming at the problem of driving safety, the application of artificial intelligence technology in safe driving behavior recognition can effectively reduce the accident rate and economic losses. Based on the presence of interference signals such as spatiotemporal background mixed signals in the driving monitoring video sequence, the recognition accuracy of small targets such as human eyes is low. In this paper, an improved dual-stream convolutional network is proposed to recognize the safe driving behavior. Based on convolutional neural networks (CNNs), attention mechanism (AM) is integrated into a long short-term memory (LSTM) neural network structure, and the hybrid dual-stream AM-LSTM convolutional network channel is designed. The spatial stream channel uses the CNN method to extract the spatial characteristic value of video image and uses pyramid pooling instead of traditional pooling, normalizing the scale transformation. The time stream channel uses a single-shot multibox detector (SSD) algorithm to calculate the adjacent two frames of video sequence for the detection of small objects such as face and eyes. Then, AM-LSTM is used to fuse and classify dual-stream information. The self-built driving behavior video image set is built. ROC, accuracy rate, and loss function experiments are carried out in the FDDB database, VOT100 data set, and self-built video image set, respectively. Compared with CNN, SSD, IDT, and dual-stream recognition methods, the accuracy rate of this method can be improved by at least 1.4%, and the average absolute error in four video sequences can be improved by more than 2%. On the contrary, in the self-built image set, the recognition rate of doze reaches 68.3%, which is higher than other methods. The experimental results show that this method has good recognition accuracy and practical application value.

show abstract

Novel deep learning model for facial expression recognition based on maximum boosted CNN and LSTM

Cited by 41 publications

References 44 publications

Multimodal Emotion Recognition using Deep Learning

Multimodal Emotion Recognition using Deep Learning

Applications of Recurrent Neural Network for Biometric Authentication & Anomaly Detection

Research on Safe Driving Evaluation Method Based on Machine Vision and Long Short-Term Memory Network

Contact Info

Product

Resources

About