A novel dual-modal emotion recognition algorithm with fusing hybrid features of audio signal and speech context

Abstract

With regard to human–machine interaction, accurate emotion recognition is a challenging problem. In this paper, efforts were taken to explore the possibility to complete the feature abstraction and fusion by the homogeneous network component, and propose a dual-modal emotion recognition framework that is composed of a parallel convolution (Pconv) module and attention-based bidirectional long short-term memory (BLSTM) module. The Pconv module employs parallel methods to extract multidimensional social features and provides more effective representation capacity. Attention-based BLSTM module is utilized to strengthen key information extraction and maintain the relevance between information. Experiments conducted on the CH-SIMS dataset indicate that the recognition accuracy reaches 74.70% on audio data and 77.13% on text, while the accuracy of the dual-modal fusion model reaches 90.02%. Through experiments it proves the feasibility to process heterogeneous information within homogeneous network component, and demonstrates that attention-based BLSTM module would achieve best coordination with the feature fusion realized by Pconv module. This can give great flexibility for the modality expansion and architecture design.

Deep causal learning for robotic intelligence

“…This requires the development of modular and scalable robot systems that can adapt to different scenarios (Li et al, 2012a;Li S. et al, 2017;Majumdar et al, 2020). • Ethical and societal concerns: as robots become more advanced and autonomous, there are ethical and societal concerns to be taken into account, such as potential job displacement, privacy, and security issues (Lin et al, 2014;Xu et al, 2022).…”

Lateral interaction by Laplacian‐based graph smoothing for deep neural networks

“…We restrict the prototypes of the same class to form a topologically ordered map on pre-defined grid structure, thus the Laplacian matrix is defined as a K*K matrix. We can infer from Equation ( 12) that DCE updates all prototypes simultaneously, with the gradient of each prototype being weighted by the probability given in Equation (11). The weights of the gradients are competitive among prototypes, causing a significant number of prototypes to learn nothing because their gradients will always shrink to zero if they are initialised far away from the data points.…”

“…Nowadays, ANNs have achieved a great success in computer vision, natural language processing, and reinforcement tasks [8][9][10]. The success of all these pieces of work affirmed the importance to apply the structure of biological brain in studying machine learning [11][12][13].…”

Efficient three-dimensional point cloud object detection based on improved Complex-YOLO

“…Deep learning-related technologies are increasingly integrated into people’s daily life, and object detection algorithms ( Qi et al, 2021 ; Liu et al, 2022a , b ; Xu et al, 2022 ), as a crucial component of the autonomous driving perception layer, can create a solid foundation for behavioral judgments during autonomous driving. Although object detection algorithms based on 2D images ( Bochkovskiy et al, 2020 ; Bai et al, 2022 ; Cheon et al, 2022 ; Gromada et al, 2022 ; Long et al, 2022 ; Otgonbold et al, 2022 ; Wahab et al, 2022 ; Wang et al, 2022 ) have had a lot of success at this stage, single-view images cannot completely reflect the position pose, and motion orientation of objects in 3D space due to the lack of depth information in 2D images.…”