A Human–Computer Interaction framework for emotion recognition through time-series thermal video sequences

Nayak, Satyajit; Nagesh, B S; Routray, Aurobinda; Sarma, Monalisa

doi:10.1016/j.compeleceng.2021.107280

Cited by 54 publications

(23 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Facial expressions, voice intonation, and body posture can convey different emotions. Therefore, recognizing different people's emotions has become a key factor affecting the development of human-computer interaction and intelligence (Samara et al, 2019;Hu et al, 2021;Nayak et al, 2021). Generally, emotional information contained in the face is the most direct and abundant.…”

Section: Introductionmentioning

confidence: 99%

New Breakthroughs and Innovation Modes in English Education in Post-pandemic Era

Shen

Guo

2022

Front. Psychol.

View full text Add to dashboard Cite

The outbreak of COVID-19 has brought drastic changes to English teaching as it has shifted from the offline mode before the pandemic to the online mode during the pandemic. However, in the post-pandemic era, there are still many problems in the effective implementation of the process of English teaching, leading to the inability of achieving better results in the quality and efficiency of English teaching and effective cultivation of students’ practical application ability. In recent years, English speaking has attracted the attention of experts and scholars. Therefore, this study constructs an interactive English-speaking practice scene based on a virtual character. A dual-modality emotion recognition method is proposed that mainly recognizes and analyzes facial expressions and physiological signals of students and the virtual character in each scene. Thereafter, the system adjusts the difficulty of the conversation according to the current state of students, toward making the conversation more conducive to the students’ understanding and gradually improving their English-speaking ability. The simulation compares nine facial expressions based on the eNTERFACE05 and CAS-PEAL datasets, which shows that the emotion recognition method proposed in this manuscript can effectively recognize students’ emotions in interactive English-speaking practice and reduce the recognition time to a great extent. The recognition accuracy of the nine facial expressions was close to 90% for the dual-modality emotion recognition method in the eNTERFACE05 dataset, and the recognition accuracy of the dual-modality emotion recognition method was significantly improved with an average improvement of approximately 5%.

show abstract

Section: Introductionmentioning

confidence: 99%

New Breakthroughs and Innovation Modes in English Education in Post-pandemic Era

Shen

Guo

2022

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…Emotions are complex processes comprised of numerous components, including feelings, body changes, cognitive reactions, behavior, and thoughts [ 41 ]. Emotion is a psycho-physiological process triggered by the conscious and unconscious perception of a situation or an object and is often associated with mood, temperament, personality, disposition, and motivation [ 42 ]. Intelligent systems providing HCII must, for example, through emotion recognition, be able to perceive the user’s emotions, produce the ability of empathy, and respond appropriately [ 30 , 43 ].…”

Section: Backgrounds and Related Workmentioning

confidence: 99%

“…Emotions, for example, can trigger some minor changes in facial blood flow with an impact on skin temperature [ 42 ] and speech [ 105 ]. EEG-based emotion recognition, for example, has become crucial in enabling the HCII [ 44 ] and has been globally accepted in many applications, such as intelligent thinking, decision-making, social communication, feeling detection, affective computing, etc.…”

Section: Backgrounds and Related Workmentioning

confidence: 99%

Sensors and Artificial Intelligence Methods and Algorithms for Human–Computer Intelligent Interaction: A Systematic Mapping Study

Šumak

Brdnik

Pušnik

2021

Sensors

View full text Add to dashboard Cite

To equip computers with human communication skills and to enable natural interaction between the computer and a human, intelligent solutions are required based on artificial intelligence (AI) methods, algorithms, and sensor technology. This study aimed at identifying and analyzing the state-of-the-art AI methods and algorithms and sensors technology in existing human–computer intelligent interaction (HCII) research to explore trends in HCII research, categorize existing evidence, and identify potential directions for future research. We conduct a systematic mapping study of the HCII body of research. Four hundred fifty-four studies published in various journals and conferences between 2010 and 2021 were identified and analyzed. Studies in the HCII and IUI fields have primarily been focused on intelligent recognition of emotion, gestures, and facial expressions using sensors technology, such as the camera, EEG, Kinect, wearable sensors, eye tracker, gyroscope, and others. Researchers most often apply deep-learning and instance-based AI methods and algorithms. The support sector machine (SVM) is the most widely used algorithm for various kinds of recognition, primarily an emotion, facial expression, and gesture. The convolutional neural network (CNN) is the often-used deep-learning algorithm for emotion recognition, facial recognition, and gesture recognition solutions.

show abstract

“…Computer vision is usually used as the basic technology of human-computer interaction [31], but it is rarely found that the performance of computer vision algorithms can be improved through human-computer collaboration. Some existing human-like automatic visual tracking methods [32][33][34] try to analyze contextual information based on human cognition and introduce part of expert experience into the tracking method.…”

Section: Comparison With State-of-the-art Trackersmentioning

confidence: 99%

Long-Term Person Tracking for Unmanned Aerial Vehicle Based on Human-Machine Collaboration

Zhou

Liu

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

A Human–Computer Interaction framework for emotion recognition through time-series thermal video sequences

Cited by 54 publications

References 17 publications

New Breakthroughs and Innovation Modes in English Education in Post-pandemic Era

New Breakthroughs and Innovation Modes in English Education in Post-pandemic Era

Sensors and Artificial Intelligence Methods and Algorithms for Human–Computer Intelligent Interaction: A Systematic Mapping Study

Long-Term Person Tracking for Unmanned Aerial Vehicle Based on Human-Machine Collaboration

Contact Info

Product

Resources

About