Deep Convolutional and Recurrent Networks for Polyphonic Instrument Classification from Monophonic Raw Audio Waveforms

Abstract

Sound Event Detection and Audio Classification tasks are traditionally addressed through time-frequency representations of audio signals such as spectrograms. However, the emergence of deep neural networks as efficient feature extractors has enabled the direct use of audio signals for classification purposes. In this paper, we attempt to recognize musical instruments in polyphonic audio by only feeding their raw waveforms into deep learning models. Various recurrent and convolutional architectures incorporating residual connections are examined and parameterized in order to build end-to-end classifiers with low computational cost and only minimal preprocessing. We obtain competitive classification scores and useful instrumentwise insight through the IRMAS test set, utilizing a parallel CNN-BiGRU model with multiple residual connections, while maintaining a significantly reduced number of trainable parameters.

Musical Instrument Identification Using Deep Learning Approach

“…The first consists of extracting a feature vector (FV) containing audio descriptors and using the baseline machine learning algorithms [12,[15][16][17][18][19][20][21][22][23][24][25][26][27]. The second is based on the 2D audio representation and a deep learning model [28][29][30][31][32][33][34][35][36][37][38][39][40][41], or a more automated version when a variational or deep softmax autoencoder is used for the audio representation retrieval [32,42]. Therefore, by employing machine learning, it is possible to implement a classifier for particular genres or instrument recognition.…”

“…Reviewing the literature that describes the classification of musical instruments, it can be seen that this has been in development for almost three decades [17,18,25,28,36,41]. These works use various sets of signals and statistical parameters for the analyzed samples, standard MPEG-7 descriptors, spectrograms, mel-frequency cepstral coefficients (MFCC), or constant-Q transform (CQT)-the basis for their operation.…”

“…For example, the cello and saxophone are often encountered in jazz music, whereas the banjo is almost exclusively associated with country music. In music genre classification, several well-known techniques have been used, such as SVM (support vector machine) [14,19,25,26,33], ANN (artificial neural networks) [24,40], etc., as well as CNN (convolutional neural networks) [28,30,[34][35][36][37][38][39][40], RNN (recurrent neural networks) [28,41], and CRNN (convolutional recurrent neural network) [31,34].…”

“…Table 1 shows an overview of various algorithms and tasks described above along with the obtained results [14][15][16][18][19][20][21][22][23][24][25][26][27][28][29][30][31][33][34][35][36][37][38][39][40][41]. A system that can listen to the musical instrument tone and recognize it (no metrics shown) Lee J., Kim T., Park J., Nam J.…”