PSLA: Improving Audio Tagging With Pretraining, Sampling, Labeling, and Aggregation

Abstract

Audio event classification is an active research area and has a wide range of applications. Since the release of AudioSet, great progress has been made in advancing the classification accuracy, which mostly comes from the development of novel model architectures and attention modules. However, we find that appropriate training techniques are equally important for building audio event classification models with AudioSet, but have not received the attention they deserve. To fill the gap, in this work, we present PSLA, a collection of training techniques that can noticeably boost the model accuracy including ImageNet pretraining, balanced sampling, data augmentation, label enhancement, model aggregation and their design choices.By training an EfficientNet with these techniques, we obtain a model that achieves a new state-of-the-art mean average precision (mAP) of 0.474 on AudioSet, outperforming the previous best system of 0.439.

HTS-AT: A hierarchical token-semantic audio transformer for sound classification and detection

“…The AudioSet contains over two million 10-sec audio samples labeled with 527 sound event classes. In this paper, we follow the same training pipeline in [11,12,14] by using the full-train set (2M samples) to our model and it evaluation set (22K samples). All samples are converted to mono as 1 channel by 32kHz rate.…”

“…We follow [11,12] to use the balance sampler, α = 0.5 mix-up [19], spectrogram masking [20] with time-mask=128 frames and frequency-mask=16 bins, and weight averaging. The HTS-AT is implemented in Pytorch and trained via the AdamW optimizer (β 1 =0.9, β 2 =0.999, eps=1e-8, de-cay=0.05) with a batch size of 128 (32 × 4) in 4 NVIDIA Tesla V-100 GPUs.…”

“…For the model design, the audio classification task is thriving based on neural-network-based models. Convolutional neural networks (CNNs) have been widely used in this field, such as DeepResNet [9], TALNet [10], PANN [11], and PSLA [12]. These models leverage CNN to capture features on the audio spectrogram, and further improve their performance through the design of the depth and breadth of the network.…”

AudioTagging Done Right: 2nd comparison of deep learning methods for environmental sound classification

“…The same group [17] conducted quality assessments of these labels ranging from 0-100%. Label enhancement [7] involves altering the original labels. CNN14 [10] achieves 0.442 mean average precision (mAP) using 128 mel bins.…”

“…Vision Transformers (ViT) [6]-in computer vision and other fields, and competitive performances are reported. Recently, AST [2] and PSLA [7] improved the SOTA performance of the AT task 3 on the AudioSet benchmark by leveraging a suite of improvements including DeiT [1] ( distilled ViT) architecture, ImageNet pretraining, data augmentations, and ensemble. However, there is still no clear "winner-takes-all" approach in audio classification tasks that can have the best performance while being efficient.…”