Data-Dependent Feature Extraction Method Based on Non-Negative Matrix Factorization for Weakly Supervised Domestic Sound Event Detection

Abstract

In this paper, feature extraction methods are developed based on the non-negative matrix factorization (NMF) algorithm to be applied in weakly supervised sound event detection. Recently, the development of various features and systems have been attempted to tackle the problems of acoustic scene classification and sound event detection. However, most of these systems use data-independent spectral features, e.g., Mel-spectrogram, log-Mel-spectrum, and gammatone filterbank. Some data-dependent feature extraction methods, including the NMF-based methods, recently demonstrated the potential to tackle the problems mentioned above for long-term acoustic signals. In this paper, we further develop the recently proposed NMF-based feature extraction method to enable its application in weakly supervised sound event detection. To achieve this goal, we develop a strategy for training the frequency basis matrix using a heterogeneous database consisting of strongly- and weakly-labeled data. Moreover, we develop a non-iterative version of the NMF-based feature extraction method so that the proposed feature extraction method can be applied as a part of the model structure similar to the modern “on-the-fly” transform method for the Mel-spectrogram. To detect the sound events, the temporal basis is calculated using the NMF method and then used as a feature for the mean-teacher-model-based classifier. The results are improved for the event-wise post-processing method. To evaluate the proposed system, simulations of the weakly supervised sound event detection were conducted using the Detection and Classification of Acoustic Scenes and Events 2020 Task 4 database. The results reveal that the proposed system has F1-score performance comparable with the Mel-spectrogram and gammatonegram and exhibits 3–5% better performance than the log-Mel-spectrum and constant-Q transform.

A Survey of Audio Enhancement Algorithms for Music, Speech, Bioacoustics, Biomedical, Industrial, and Environmental Sounds by Image U-Net

“…As found in many audio enhancement applications the vanilla architecture usually does not generate an impressive audio quality (although comparable or slightly better than the machine learning algorithms e.g. [66], [67], [68], [69], [70], and [71]) in the output. Therefore either additional layers are added, skip connections are modified, hyper parameters are adjusted or completely new architectures are proposed in many recent AE models to improve the performance.…”

“…However, the event detection tasks do not require the frequency axis restoration as the required information lies on the time axis. So in LUU-Net only time scale is restored by limited upscaling of the frequency axis by using asymmetric stride for decoder convolutional layers [70]. This results in an immense reduction in learnable parameters.…”

“…To separate the background from the foreground sound of rare events, the sound event detection model of [70] uses two U-Net architectures: 1) the conventional U-Net, and 2) a novel U-Net architecture called U-Net with Limited Upsampling (LUU-Net) which applies limited upsampling on the decoder side to restore the original time axis and only a limited frequency axis. This saves the computational cost by 40% as compared to the conventional U-Net without any information loss as the onset and offset information of the rare event lies on the time axis.…”

Classifying the Main Technology Clusters and Assignees of Home Automation Networks Using Patent Classifications

“…Although NMF is quite an old algorithm, it is considered an effective and widespread methodology as it is still employed in multiple areas of interest, such as sound event detection [29], speech recognition [30], text mining [31][32][33], image analysis [34,35], security and privacy [36] and community detection [37]. In many of these applications, NMF is used as a tool for classification, filtering, dimensionality reduction as well as data clustering, which is the core concept of this study.…”

Machine Learning Methods with Noisy, Incomplete or Small Datasets

“…This paper illustrates the ability of these decomposition models to impute missing features, denoising and to artificially generate additional data samples (data augmentation) with examples to the brain-computer interface (BCI) and epileptic EEG analysis, among others. In [9], Lee et al (South Korea) developed feature extraction methods based on the non-negative matrix factorization (NMF) algorithm and it is applied in weakly supervised sound event detection.…”