Robust Voice Activity Detection Using a Masked Auditory Encoder Based Convolutional Neural Network

Abstract

Voice activity detection (VAD) based on deep learning has achieved remarkable success. However, when the traditional features (e.g., raw waveforms and MFCCs) are directly fed to the deep neural network model, the performance decreases because of noise interference. Here, we propose a robust VAD approach using a masked auditory encoder based convolutional neural network (M-AECNN). First, we analyze the effectiveness of using auditory features as deep learning encoder. These features can roughly simulate the transmission of sound to human inner-ear hair cells; thus, they are more robust than the raw waveform and frequency domain features designed as encoders. Second, similar to the human ear's masking effect for different speech frequencies, the proposed auditory encoder can further improve the robustness of VAD by increasing the gain for cleaner speech frequencies. Extensive experimental results demonstrate that this approach achieves about 10.5% absolute improvement in the area under the curve on the AURORA-2J dataset compared with a VAD method based on a CNN and MFCCs.

Teachable Machine: Deteksi Dialek Sumba Timur (Kambera) Menggunakan Layanan Open Source

“…CNN has been applied to research in various fields with different case studies of voice detections. This research includes voice detection to classify voices [22], [23], detect depressive or emotional voices [24], [25], detect music [26], realize voice-based security systems [27], [28], detect language phonemes [29], [30], detect disease by sounds [31]- [34], and identify animal voices [35], [36]. There are many other research case studies that can be found and used as ongoing future research.…”

Supervised Contrastive Learning for Voice Activity Detection

“…Early VAD research works were conducted based on statistical models [1,2]. Recently, deep learning-based VAD models leveraging various neural network models have been actively explored, including convolutional neural networks (CNN) [3][4][5][6][7], recurrent neural networks (RNN), and long short-term memory (LSTM) networks [8][9][10][11]. In particular, Jia et al and Kopuklu et al [4,5] proposed compact VAD models suitable for use in a limited hardware resource environment.…”