Multimodal Emotion Recognition Algorithm for Artificial Intelligence Information System

Weng, Yabin; Lin, Feiting

doi:10.1155/2022/9236238

Cited by 6 publications

(3 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the latest social networks, more and more multimodal emotion analysis is being conducted on videos through text, speech, and rich facial expressions [ 44 ]. However, multimodal research has a high standard for the size and diversity of corpora, and currently, available datasets are not enough to support broader research.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Emotion Recognition Based on Cascaded Multichannel and Hierarchical Fusion

Liu

Huang

2023

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Humans express their emotions in a variety of ways, which inspires research on multimodal fusion-based emotion recognition that utilizes different modalities to achieve information complementation. However, extracting deep emotional features from different modalities and fusing them remain a challenging task. It is essential to exploit the advantages of different extraction and fusion approaches to capture the emotional information contained within and across modalities. In this paper, we present a novel multimodal emotion recognition framework called multimodal emotion recognition based on cascaded multichannel and hierarchical fusion (CMC-HF), where visual, speech, and text signals are simultaneously utilized as multimodal inputs. First, three cascaded channels based on deep learning technology perform feature extraction for the three modalities separately to enhance deeper information extraction ability within each modality and improve recognition performance. Second, an improved hierarchical fusion module is introduced to promote intermodality interactions of three modalities and further improve recognition and classification accuracy. Finally, to validate the effectiveness of the designed CMC-HF model, some experiments are conducted to evaluate two benchmark datasets, IEMOCAP and CMU-MOSI. The results show that we achieved an almost 2%∼3.2% increase in accuracy of the four classes for the IEMOCAP dataset as well as an improvement of 0.9%∼2.5% in the average class accuracy for the CMU-MOSI dataset when compared to the existing state-of-the-art methods. The ablation experimental results indicate that the cascaded feature extraction method and the hierarchical fusion method make a significant contribution to multimodal emotion recognition, suggesting that the three modalities contain deeper information interactions of both intermodality and intramodality. Hence, the proposed model has better overall performance and achieves higher recognition efficiency and better robustness.

show abstract

Section: Related Workmentioning

confidence: 99%

Multimodal Emotion Recognition Based on Cascaded Multichannel and Hierarchical Fusion

Liu

Huang

2023

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Computer vision algorithms are employed to analyze facial features and dynamics, mapping them to specific emotional states (Wang, 2017). Similarly, natural language processing techniques are utilized to decode emotional nuances in spoken or written language (Weng & Lin, 2022), As shown in Error! Reference source not found..…”

Section: Emotion Recognitionmentioning

confidence: 99%

Emotion Recognition in Psychology of Human-robot Interaction

Zhao

2023

Psychomachina

View full text Add to dashboard Cite

The field of Human-Robot Interaction (HRI) has garnered significant attention in recent years, with researchers and practitioners seeking to understand the psychological aspects underlying the interactions between humans and robots. One crucial area of focus within HRI is the psychology of emotion recognition, which plays a fundamental role in shaping the dynamics of human-robot interaction. This paper provides an overview of the background of psychology in the context of human-robot interaction, emphasizing the significance of understanding human emotions in this domain. The concept of emotion recognition, a key component of human psychology, is explored in detail, highlighting its relevance in the context of human-robot interaction. Emotion recognition allows robots to perceive and interpret human emotions, enabling them to respond appropriately and enhance the quality of interaction. The role of emotion recognition in HRI is examined from a psychological standpoint, shedding light on its implications for the design and development of effective human-robot interfaces. Furthermore, this paper delves into the application of machine learning techniques for emotion recognition in the context of human-robot interaction. Machine learning algorithms have shown promise in enabling robots to recognize and respond to human emotions, thereby contributing to more natural and intuitive interactions. The utilization of machine learning in emotion recognition reflects the intersection of psychology and technological advancements in the field of HRI. Finally, the challenges associated with emotion recognition in HRI are discussed, encompassing issues such as cross-cultural variations in emotional expression, individual differences, and the ethical implications of emotion detection. Addressing these challenges is pivotal in advancing the understanding and implementation of emotion recognition in human-robot interaction, underscoring the interdisciplinary nature of this endeavor. In conclusion, this paper underscores the critical role of emotion recognition in the psychology of human-robot interaction, emphasizing its potential to revolutionize the way humans and robots engage with each other. By integrating insights from psychology, machine learning, and technology, advancements in emotion recognition have the potential to pave the way for more empathetic and responsive human-robot interactions, offering new avenues for research and practical applications in this burgeoning field.

show abstract

“…Wireless Communications and Mobile Computing has retracted the article titled "Multimodal Emotion Recognition Algorithm for Artificial Intelligence Information System" [1] due to concerns that the peer review process has been compromised.…”

mentioning

confidence: 99%

Retracted: Multimodal Emotion Recognition Algorithm for Artificial Intelligence Information System

Computing¹

2023

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Multimodal Emotion Recognition Algorithm for Artificial Intelligence Information System

Cited by 6 publications

References 30 publications

Multimodal Emotion Recognition Based on Cascaded Multichannel and Hierarchical Fusion

Multimodal Emotion Recognition Based on Cascaded Multichannel and Hierarchical Fusion

Emotion Recognition in Psychology of Human-robot Interaction

Retracted: Multimodal Emotion Recognition Algorithm for Artificial Intelligence Information System

Contact Info

Product

Resources

About