Two-Dimensional Convolutional Recurrent Neural Networks for Speech Activity Detection

Abstract

Speech Activity Detection (SAD) plays an important role in mobile communications and automatic speech recognition (ASR). Developing efficient SAD systems for real-world applications is a challenging task due to the presence of noise. We propose a new approach to SAD where we treat it as a twodimensional multilabel image classification problem. To classify the audio segments, we compute their Short-time Fourier Transform spectrograms and classify them with a Convolutional Recurrent Neural Network (CRNN), traditionally used in image recognition. Our CRNN uses a sigmoid activation function, max-pooling in the frequency domain, and a convolutional operation as a moving average filter to remove misclassified spikes. On the development set of Task 1 of the 2019 Fearless Steps Challenge, our system achieved a decision cost function (DCF) of 2.89%, a 66.4% improvement over the baseline. Moreover, it achieved a DCF score of 3.318% on the evaluation dataset of the challenge, ranking first among all submissions.

EML Online Speech Activity Detection for the Fearless Steps Challenge Phase-III

“…The buffer window lengths for speech and non-speech segments (Lsp and Lnsp in Eqs. [3][4][5] are set to 30 and 60 segments corresponding to 3 and 6 sec, respectively. The adaptation weights α and β (Eqs.…”

“…The adaptation weights α and β (Eqs. [3][4][5] are set to 0.4 and 0.1, respectively. The decision threshold for the non-adaptive system (θ M ) is set to 0.25.…”

“…Various SAD algorithms have previously been proposed for phases I and II in this challenge, e.g., [4][5][6][7][8][9]. Most of the proposed algorithms and models are based on advanced deep learning techniques, mainly Convolutional and Recurrent Neural Networks, learning the statistical properties of speech and non-speech frames within the available training data.…”

Unsupervised Adaptation of Deep Speech Activity Detection Models to Unseen Domains

“…A number of examples of the use of CRNN models in audio processing can be found in the literature [15,16]. CRNN models have been also applied to the SAD task with relevant results [17].…”

Robust Voice Activity Detection Using a Masked Auditory Encoder Based Convolutional Neural Network

“…However, direct use of these DL systems in speech/non-speech classification often cannot fully mine speech information under noisy conditions, which reduces VAD robustness. The most important reason for the above problem is that, the encoders of the network are not robust, that is, they are designed to use energy levels, zerocrossings, log-spectrograms, Mel-frequency cepstral coefficients (MFCCs), and raw waveforms [6,10,13].…”