Hierarchical Pooling Structure for Weakly Labeled Sound Event Detection

He, Kebin; Shen, Yuhan; Zhang, Weiqiang

doi:10.21437/interspeech.2019-2049

Cited by 8 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Model is trained with ADAM optimizer with an initial learning rate of 0.001 for 30 epochs, and the learning rate is reduced by half every 10 epochs. We chose Classification F1 Dev (%) Eval (%) Xu et al [5] [14] 46.8 N/A Yan et al [6] 51.3 55.1 He et al [12] 46.5 53.4 Ours 49.9 49.4 Table 3: Results on DCASE 2017 task 4B: weakly supervised AED for smart cars the best weights out of 30 epochs based on classification F1 on the dev set. The batch size is set to 48 due to GPU memory constraints.…”

Section: Dcase 2018 Taskmentioning

confidence: 99%

“…The winner of DCASE 2017 challenge used an ensemble of CNNs with various lengths of analysis windows for multiple input scaling [11]. He et al [12] proposed a hierarchical pooling structure to improve the performance of CRNN. The effect of different pooling/attention methods on AED and audio tagging also have been analyzed in previous works [13,14,15].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Joint Framework for Audio Tagging and Weakly Supervised Acoustic Event Detection Using DenseNet with Global Average Pooling

Kao¹,

Shi²,

Sun³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

This paper proposes a network architecture mainly designed for audio tagging, which can also be used for weakly supervised acoustic event detection (AED). The proposed network consists of a modified DenseNet as the feature extractor, and a global average pooling (GAP) layer to predict frame-level labels at inference time. This architecture is inspired by the work proposed by Zhou et al., a well-known framework using GAP to localize visual objects given image-level labels. While most of the previous works on weakly supervised AED used recurrent layers with attention-based mechanism to localize acoustic events, the proposed network directly localizes events using the feature map extracted by DenseNet without any recurrent layers. In the audio tagging task of DCASE 2017, our method significantly outperforms the state-of-the-art method in F1 score by 5.3% on the dev set, and 6.0% on the eval set in terms of absolute values. For weakly supervised AED task in DCASE 2018, our model outperforms the state-of-the-art method in event-based F1 by 8.1% on the dev set, and 0.5% on the eval set in terms of absolute values, by using data augmentation and tri-training to leverage unlabeled data.

show abstract

Section: Dcase 2018 Taskmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Joint Framework for Audio Tagging and Weakly Supervised Acoustic Event Detection Using DenseNet with Global Average Pooling

Kao¹,

Shi²,

Sun³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

show abstract

“…For example, Kong et al [12] proposed an attention model as a pooling function, which is achieved by a weighted sum of the results over frames. He et al [13] proposed a hierarchical attention pooling structure. By assigning larger weights to the instance corresponding to the sound events, these methods could dynamically determine the contribution of each instance.…”

Section: Introductionmentioning

confidence: 99%

Gated Multi-Head Attention Pooling for Weakly Labelled Audio Tagging

Hong¹,

Zou²,

Wang³

2020

Interspeech 2020

View full text Add to dashboard Cite

Multiple instance learning (MIL) has recently been used for weakly labelled audio tagging, where the spectrogram of an audio signal is divided into segments to form instances in a bag, and then the low-dimensional features of these segments are pooled for tagging. The choice of a pooling scheme is the key to exploiting the weakly labelled data. However, the traditional pooling schemes are usually fixed and unable to distinguish the contributions, making it difficult to adapt to the characteristics of the sound events. In this paper, a novel pooling algorithm is proposed for MIL, named gated multi-head attention pooling (GMAP), which is able to attend to the information of events from different heads at different positions. Each head allows the model to learn information from different representation subspaces. Furthermore, in order to avoid the redundancy of multi-head information, a gating mechanism is used to fuse individual head features. The proposed GMAP increases the modeling power of the single-head attention with no computational overhead. Experiments are carried out on Audioset, which is a large-scale weakly labelled dataset, and show superior results to the non-adaptive pooling and the vanilla attention pooling schemes.

show abstract

“…Furthermore, the RNN, which can remember previous inputs through time, also performs well due to the time-series characteristics of the audio signal. Recently, the CNN and CRNN models with additional techniques, including the modified CNN [15] and pooling methods [16], were proposed. Furthermore, further studies combined with other tasks such as sound event detection and segmentation using the weakly labeled data [17] and joint sound event detection and localization [18] were proposed.In contrast, in the speech recognition domain, the use of the LMFB as a feature vector and the CNN, RNN, and CRNN as the classifiers for the acoustic model are similar to the SED, but studies of integrating a preprocessor, such as acoustic beamforming or dereverberation with the acoustic model using multi-channel audio signals, have been actively conducted to improve recognition accuracy [19][20][21].…”

mentioning

confidence: 99%

mentioning

confidence: 99%

Joint Optimization of Deep Neural Network-Based Dereverberation and Beamforming for Sound Event Detection in Multi-Channel Environments

Noh

Chang

2020

Sensors

View full text Add to dashboard Cite

In this paper, we propose joint optimization of deep neural network (DNN)-supported dereverberation and beamforming for the convolutional recurrent neural network (CRNN)-based sound event detection (SED) in multi-channel environments. First, the short-time Fourier transform (STFT) coefficients are calculated from multi-channel audio signals under the noisy and reverberant environments, which are then enhanced by the DNN-supported weighted prediction error (WPE) dereverberation with the estimated masks. Next, the STFT coefficients of the dereverberated multi-channel audio signals are conveyed to the DNN-supported minimum variance distortionless response (MVDR) beamformer in which DNN-supported MVDR beamforming is carried out with the source and noise masks estimated by the DNN. As a result, the single-channel enhanced STFT coefficients are shown at the output and tossed to the CRNN-based SED system, and then, the three modules are jointly trained by the single loss function designed for SED. Furthermore, to ease the difficulty of training a deep learning model for SED caused by the imbalance in the amount of data for each class, the focal loss is used as a loss function. Experimental results show that joint training of DNN-supported dereverberation and beamforming with the SED model under the supervision of focal loss significantly improves the performance under the noisy and reverberant environments.

show abstract

Hierarchical Pooling Structure for Weakly Labeled Sound Event Detection

Cited by 8 publications

References 7 publications

A Joint Framework for Audio Tagging and Weakly Supervised Acoustic Event Detection Using DenseNet with Global Average Pooling

A Joint Framework for Audio Tagging and Weakly Supervised Acoustic Event Detection Using DenseNet with Global Average Pooling

Gated Multi-Head Attention Pooling for Weakly Labelled Audio Tagging

Joint Optimization of Deep Neural Network-Based Dereverberation and Beamforming for Sound Event Detection in Multi-Channel Environments

Contact Info

Product

Resources

About