Optimal feature selection based speech emotion recognition using two‐stream deep convolutional neural network

Abstract

Speech signal processing is an active area of research, the most dominant source of exchanging information among human beings, and the best way for human–computer interaction (HCI). Human behavior assessments and emotion recognition from a speech signal, such as speech emotion recognition (SER) is an emerging HCI area of exploration with various real time claims. The performance of an efficient SER system depends on feature learning, which include salient and discriminative information such as high‐level deep features. In this paper, we proposed a two‐stream deep convolutional neural network with an iterative neighborhood component analysis (INCA) to learn mutually spatial‐spectral features and select the most discriminative optimal features for the final prediction. Our model is composed of two channels, and each channel is associated with the convolutional neural network structure to extract cues from the oral signals. The first channel extracts feature from the spectral domain, and the second channel extracts features from the spatial domain, which are then fused and fed to the INCA to remove the severance and select the optimal features for the final model training. The joint refine features are passed from the fully connected network with a softmax classifier to yield the predictions of the different emotions. We trained our proposed system using three benchmarks, which included the EMO‐DB, SAVEE, and RAVDESS emotional speech corpora, and we tested the prediction performance to secure 95%, 82%, and 85% recognition rates. The performance of the system shows the effectiveness and significance of the proposed system.

A robust packet‐dropping covert channel for mobile intelligent terminals

“…Codeword HASH CRC , and bl is given in (5), where bl corresponds to the size of the message block D i . Since the codeword in the inter-codeword verification resetting group lacks the covert message section, the capacity ρ is evaluated on average and is expressed by (6). Finally, the throughput of the CTC is calculated based on ρ and ψ, as shown in (7).…”

“…[1][2][3] The deployment of Voice over Long Term Evolution (VoLTE) for next-generation audio and video communication over mobile networks has significantly increased the available data rates and quality of service (QoS). [4][5][6] The core network of this technology is a packet-switched network, that is also part of the 5G standard. In recent years, some research efforts have focused on constructing covert timing channels (CTCs) over wireless and VoLTE networks.…”

“…Covert channels can be an effective and efficient means of transmitting confidential information in untrusted IoT networks 1–3 . The deployment of Voice over Long Term Evolution (VoLTE) for next‐generation audio and video communication over mobile networks has significantly increased the available data rates and quality of service (QoS) 4–6 . The core network of this technology is a packet‐switched network, that is also part of the 5G standard.…”

Document images forgery localization using a two‐stream network

“…Considering the excellent performance of deep learning technologies, 30,31 the deep learning is utilized to capture forgery traces. Relying on the deep analysis of forgery methods, different traces have been utilized for forgery localization, such as noise level, 3,5,32,33 unnatural boundaries, 34 contrast/brightness inconsistencies, [35][36][37] pixels periodic correlation caused by interpolation, 1,3,19 and so on.…”

Deep Neural Network with Data Cropping Algorithm for Absorptive Frequency‐Selective Transmission Metasurface

“…[27] It has been widely used in image recognition [28,29] and speech recognition. [30] Recently, the inverse design of many metasurface structures has been realized by utilizing DNN. [31][32][33][34] In these studies, most DNNs adopted inverse design to obtain suitable parameters according to the design goal.…”