Comparison of Pre-Trained CNNs for Audio Classification Using Transfer Learning

Abstract

The paper investigates retraining options and the performance of pre-trained Convolutional Neural Networks (CNNs) for sound classification. CNNs were initially designed for image classification and recognition, and, at a second phase, they extended towards sound classification. Transfer learning is a promising paradigm, retraining already trained networks upon different datasets. We selected three ‘Image’- and two ‘Sound’-trained CNNs, namely, GoogLeNet, SqueezeNet, ShuffleNet, VGGish, and YAMNet, and applied transfer learning. We explored the influence of key retraining parameters, including the optimizer, the mini-batch size, the learning rate, and the number of epochs, on the classification accuracy and the processing time needed in terms of sound preprocessing for the preparation of the scalograms and spectrograms as well as CNN training. The UrbanSound8K, ESC-10, and Air Compressor open sound datasets were employed. Using a two-fold criterion based on classification accuracy and time needed, we selected the ‘champion’ transfer-learning parameter combinations, discussed the consistency of the classification results, and explored possible benefits from fusing the classification estimations. The Sound CNNs achieved better classification accuracy, reaching an average of 96.4% for UrbanSound8K, 91.25% for ESC-10, and 100% for the Air Compressor dataset.

Interpretation of lung disease classification with light attention connected module

“…VGGish includes a deep audio embedding mode and is the proposed method for classifying audio from YouTube videos. Pre-trained VGGish is often used for audio classification [40] . A characteristic of the model structure is that several feature extractions are performed using the four block structures combined by convolution and max pooling.…”

Empirical Mode Decomposition-Based Feature Extraction for Environmental Sound Classification

“…Similarly, for the low-complexity acoustic scene classification dataset, the leading system uses resnet with a receptive field. For Urbansound8k, different systems are proposed [ 16 , 54 ]. These systems use feature pre-processing and post processing, transfer learning, and other methods to enhance the accuracy of the system.…”

“…In classification systems, the Mel filter bank energies are extracted using a fast Fourier transform-based algorithm to generate Mel spectrograms. Whether these systems are trained from the scratch using time–frequency representation of sounds [ 6 , 12 , 13 ] or if transfer learning is used to retrain systems trained on images to perform sound classification [ 5 , 14 , 15 , 16 ], they employ Fourier transform for feature extraction. However, there are some crucial restrictions to performing Fourier spectral analysis, which makes Fourier transform valid under extremely general conditions [ 17 , 18 ].…”

Transfer learning for the efficient detection of COVID-19 from smartphone audio data

“…Leveraging transfer learning with CNNs for audio classification problems is studied in some papers. In [19] , the usage of both image and sound CNNs is studied. In the former case, data samples are image-based sound representations such as spectrograms.…”

“…In order to evaluate the effectiveness of the transfer learning approaches in our specific application scenario, we take into account 3 state-of-the-art deep learning models for audio classification: YAMNET, VGGish, and L 3 -Net [34] . These models mainly differ in the network’s architecture and training approach.…”