MTF-CRNN: Multiscale Time-Frequency Convolutional Recurrent Neural Network for Sound Event Detection

Abstract

To reduce neural network parameter counts and improve sound event detection performance, we propose a multiscale time-frequency convolutional recurrent neural network (MTF-CRNN) for sound event detection. Our goal is to improve sound event detection performance and recognize target sound events with variable duration and different audio backgrounds with low parameter counts. We exploit four groups of parallel and serial convolutional kernels to learn high-level shift-invariant features from the time and frequency domains of acoustic samples. A two-layer bidirectional gated recurrent unit is used to capture the temporal context from the extracted high-level features. The proposed method is evaluated on two different sound event datasets. Compared to that of the baseline method and other methods, the performance is greatly improved as a single model with low parameter counts without pretraining. On the TUT Rare Sound Events 2017 evaluation dataset, our method achieved an error rate (ER) of 0.09±0.01, which was an improvement of 83% compared with the baseline. On the TAU Spatial Sound Events 2019 evaluation dataset, our system achieved an ER of 0.11±0.01, a relative improvement over the baseline of 61%, and F1 and ER values that are better than those of the development dataset. Compared to the state-of-the-art methods, our proposed network achieves competitive detection performance with only one-fifth of the network parameter counts. INDEX TERMS pattern recognition, sound event detection, multiscale learning, time-frequency transform, convolutional recurrent neural network

Gun identification from gunshot audios for secure public places using transformer learning

“…Zhang et al 14 propose an AED module called Multi-Scale Time-Frequency Attention (MTFA) it informs the model where to focus along the time and frequency axes by collecting data at different resolutions, which has not been taken care in the past. Zhang et al 15 and Shen et al 16 proposed a a multiscale time-frequency convolutional recurrent neural network (MTF-CRNN) for sound event detection 15 .…”

MLAN: Multi-Level Attention Network

“…In [39], the Multi-level Convolutional Pyramid Semantic Fusion (MCPSF) framework was proposed to integrate multi-level semantic features extracted by bag-of-visual-words (BoVW) model and convolutional neural network (CNN) model. Zhang et al [40] proposed a Multi-scale Time-Frequency Convolutional Recurrent Neural Network (MTF-CRNN) for sound time frequence map detection to improve sound event detection performance. Ding et al [41] proposed an Adaptive Multi-scale Detection (AdaMD) method, based on the hourglass neural network and the Gated Recurrent Unit (GRU) module, to extract different scale characteristics of time-frequency map.…”

SELD U-Net: Joint Optimization of Sound Event Localization and Detection With Noise Reduction

“…Gaussian mixture models [6] and hidden Markov models [7] were initially used. Following the advancement of deep learning algorithms, SED methods utilizing convolutional neural networks (CNNs) and recurrent neural networks (RNNs) were introduced [8,9,10,11]. SED performs the functions of the human auditory system in several industries, including audio surveillance [12] and social welfare [13].…”