Automated emotion recognition based on higher order statistics and deep learning algorithm

Sharma, Rahul; Pachori, Ram Bilas; Sircar, Pradip

doi:10.1016/j.bspc.2020.101867

Cited by 135 publications

(47 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The effectiveness of their solution using CNN + LSTM was about 70%, depending on the technology. Work related to the automated system for the recognition of emotions, based on higher-order statistics and the deep learning algorithm was presented by Sharma et al [ 21 ]. Their solution reached an average of around 87%.…”

Section: Discussionmentioning

confidence: 99%

“…Ding et al [ 12 ] proposed a model for recognising the unsafe actions of workers, using a convolution neural network (CNN) and long, short-term memory (LSTM). Sharma et al [ 21 ] also used the LSTM-based, deep learning technique to obtain emotion variations, based on the data from EEG signals. Rude et al [ 22 ] investigated the feasibility of using hidden Markov models and naive Bayes K-means for recognising worker activity in manufacturing processes using the Kinect sensor.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Deep Learning-Based Model for the Automated Assessment of the Activity of a Single Worker

Patalas‐Maliszewska

Halikowski

2020

Sensors

View full text Add to dashboard Cite

Nowadays, it is necessary to verify the accuracy of servicing work, undertaken by new employees, within a manufacturing company. A gap in the research has been observed in effective methods to automatically evaluate the work of a newly employed worker. The main purpose of the study is to build a new, deep learning model, in order to automatically assess the activity of the single worker. The proposed approach integrates the methods known as CNN, CNN + SVM, CNN + R-CNN, four new algorithms and a piece of work from a selected company, using this as an own-created dataset, in order to create a solution enabling assessment of the activity of single workers. Data were collected from an operational manufacturing cell without any guided or scripted work. The results reveal that the model developed is able to accurately detect the correctness of the work process. The model’s accuracy mostly exceeds current state-of-the-art methods for detecting work activities in manufacturing. The proposed two-stage approach, firstly, assigning the appropriate graphic instruction to a given employee’s activity using CNN and then using R-CNN to isolate the object from the reference frames, yields 94.01% and 73.15% accuracy of identification, respectively.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Deep Learning-Based Model for the Automated Assessment of the Activity of a Single Worker

Patalas‐Maliszewska

Halikowski

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…One early decision when working with EEG for emotion recognition is related to the number of electrodes to use. In the literature, this number varies from only 2 electrodes [ 13 , 14 ] to a maximum of 64 electrodes [ 15 , 16 ], with the most common value revolving around 32 [ 2 , 17 , 18 , 19 , 20 ]. Usually, the placement of the electrodes in the scalp is done according to the international 10–20 system.…”

Section: Background and Related Workmentioning

confidence: 99%

Predicting Exact Valence and Arousal Values from EEG

Galvão

Alarcão

Fonseca

2021

Sensors

View full text Add to dashboard Cite

Recognition of emotions from physiological signals, and in particular from electroencephalography (EEG), is a field within affective computing gaining increasing relevance. Although researchers have used these signals to recognize emotions, most of them only identify a limited set of emotional states (e.g., happiness, sadness, anger, etc.) and have not attempted to predict exact values for valence and arousal, which would provide a wider range of emotional states. This paper describes our proposed model for predicting the exact values of valence and arousal in a subject-independent scenario. To create it, we studied the best features, brain waves, and machine learning models that are currently in use for emotion classification. This systematic analysis revealed that the best prediction model uses a KNN regressor (K = 1) with Manhattan distance, features from the alpha, beta and gamma bands, and the differential asymmetry from the alpha band. Results, using the DEAP, AMIGOS and DREAMER datasets, show that our model can predict valence and arousal values with a low error (MAE < 0.06, RMSE < 0.16) and a strong correlation between predicted and expected values (PCC > 0.80), and can identify four emotional classes with an accuracy of 84.4%. The findings of this work show that the features, brain waves and machine learning models, typically used in emotion classification tasks, can be used in more challenging situations, such as the prediction of exact values for valence and arousal.

show abstract

“…Due to the richness of human emotion expression, emotion recognition can use different combinations of multiple emotion expression materials to make judgments. In the historical process of emotion recognition research, scholars have researched from single-modal modeling (Edgar et al, 2020;Gosztolya, 2020;Panda et al, 2020), hybrid multi-modal modeling (Ayata et al, 2020;Zhang et al, 2020), to complex deep neural networks (Abdulsalam et al, 2019;Aouani and Ayed, 2020;Sharma et al, 2020). The relationship between human emotion and its multiple expressions is gradually being unearthed.…”

Section: Introductionmentioning

confidence: 99%

A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning

Chen

Huang

et al. 2021

Front. Psychol.

View full text Add to dashboard Cite

Emotional design is an important development trend of interaction design. Emotional design in products plays a key role in enhancing user experience and inducing user emotional resonance. In recent years, based on the user's emotional experience, the design concept of strengthening product emotional design has become a new direction for most designers to improve their design thinking. In the emotional interaction design, the machine needs to capture the user's key information in real time, recognize the user's emotional state, and use a variety of clues to finally determine the appropriate user model. Based on this background, this research uses a deep learning mechanism for more accurate and effective emotion recognition, thereby optimizing the design of the interactive system and improving the user experience. First of all, this research discusses how to use user characteristics such as speech, facial expression, video, heartbeat, etc., to make machines more accurately recognize human emotions. Through the analysis of various characteristics, the speech is selected as the experimental material. Second, a speech-based emotion recognition method is proposed. The mel-Frequency cepstral coefficient (MFCC) of the speech signal is used as the input of the improved long and short-term memory network (ILSTM). To ensure the integrity of the information and the accuracy of the output at the next moment, ILSTM makes peephole connections in the forget gate and input gate of LSTM, and adds the unit state as input data to the threshold layer. The emotional features obtained by ILSTM are input into the attention layer, and the self-attention mechanism is used to calculate the weight of each frame of speech signal. The speech features with higher weights are used to distinguish different emotions and complete the emotion recognition of the speech signal. Experiments on the EMO-DB and CASIA datasets verify the effectiveness of the model for emotion recognition. Finally, the feasibility of emotional interaction system design is discussed.

show abstract

Automated emotion recognition based on higher order statistics and deep learning algorithm

Cited by 135 publications

References 30 publications

A Deep Learning-Based Model for the Automated Assessment of the Activity of a Single Worker

A Deep Learning-Based Model for the Automated Assessment of the Activity of a Single Worker

Predicting Exact Valence and Arousal Values from EEG

A Novel User Emotional Interaction Design Model Using Long and Short-Term Memory Networks and Deep Learning

Contact Info

Product

Resources

About