Spectrogram based multi-task audio classification

Zeng, Yuni; Mao, Hua; Peng, Dezhong; Zhang, Yi

doi:10.1007/s11042-017-5539-3

Cited by 135 publications

(53 citation statements)

References 25 publications

(40 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Haytham et al [38] described a neural network and RNN-based technique for SER using spectrograms of frames to train DNN, which has a high computational coast and did not achieve high accuracy. Tripathi et al [43] described the deep learning models for SER using transcript and phoneme. To train the different models with different features to increase the accuracy up to 71%, but they used same architecture, which is used for computer vision-related tasks.…”

Section: Discussionmentioning

confidence: 99%

“…It represents the significance and efficiency of the proposed DSCNN model on the RAVDESS dataset, which outperformed results in SER. Yuni et al [43] presented a spectrogram-based CNN model for multi-class audio classification on the combination of two models to achieve 64.48%, accuracy in multitask SER. Jalal et al [44] and Anjali et al [45] used the log spectrogram and spectral feature to recognize the emotion in speech data with 68% and 75%, accuracy, respectively.…”

Section: Discussionmentioning

confidence: 99%

“…Zeng et al [43] Spectrograms --64.48% Jalal et al [44] log-spectrogram -69.4% 68.10% Bhavan et al [45] spectral features --75.69% Proposed model Raw_Spectrograms 68% 61% 70.00% Proposed model Clean_Spectrograms 80% 79% 79.5% Table 10. Computational comparison of the suggested DSCNN model with other baseline CNNs models.…”

Section: Input Weighted Accuracy Unweighted Accuracy Accuracymentioning

confidence: 99%

See 2 more Smart Citations

A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition

Mustaqeem

Kwon

2019

Sensors

240

View full text Add to dashboard Cite

Speech is the most significant mode of communication among human beings and a potential method for human-computer interaction (HCI) by using a microphone sensor. Quantifiable emotion recognition using these sensors from speech signals is an emerging area of research in HCI, which applies to multiple applications such as human-reboot interaction, virtual reality, behavior assessment, healthcare, and emergency call centers to determine the speaker's emotional state from an individual's speech. In this paper, we present major contributions for; (i) increasing the accuracy of speech emotion recognition (SER) compared to state of the art and (ii) reducing the computational complexity of the presented SER model. We propose an artificial intelligence-assisted deep stride convolutional neural network (DSCNN) architecture using the plain nets strategy to learn salient and discriminative features from spectrogram of speech signals that are enhanced in prior steps to perform better. Local hidden patterns are learned in convolutional layers with special strides to down-sample the feature maps rather than pooling layer and global discriminative features are learned in fully connected layers. A SoftMax classifier is used for the classification of emotions in speech. The proposed technique is evaluated on Interactive Emotional Dyadic Motion Capture (IEMOCAP) and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) datasets to improve accuracy by 7.85% and 4.5%, respectively, with the model size reduced by 34.5 MB. It proves the effectiveness and significance of the proposed SER technique and reveals its applicability in real-world applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Input Weighted Accuracy Unweighted Accuracy Accuracymentioning

confidence: 99%

See 1 more Smart Citation

A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition

Mustaqeem

Kwon

2019

Sensors

240

View full text Add to dashboard Cite

show abstract

“…In Oramas [20], a hybrid approach was presented that combined diverse modalities (album cover images, reviews, and audio tracks) for multi-label music genre classification by applying deep learning techniques appropriate for each modality, an approach that outperformed the single-modality methods. Finally, it should be mentioned that many methods in machine learning are also proposed for the human voice classification task: emotion recognition [21], English accent classification, and gender classification [22], to name a few.…”

Section: Introductionmentioning

confidence: 99%

Spectrogram Classification Using Dissimilarity Space

Nanni¹,

Rigo²,

Lumini

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

In this work, we combine a Siamese neural network and different clustering techniques to generate a dissimilarity space that is then used to train an SVM for automated animal audio classification. The animal audio datasets used are (i) birds and (ii) cat sounds, which are freely available. We exploit different clustering methods to reduce the spectrograms in the dataset to a number of centroids that are used to generate the dissimilarity space through the Siamese network. Once computed, we use the dissimilarity space to generate a vector space representation of each pattern, which is then fed into an support vector machine (SVM) to classify a spectrogram by its dissimilarity vector. Our study shows that the proposed approach based on dissimilarity space performs well on both classification problems without ad-hoc optimization of the clustering methods. Moreover, results show that the fusion of CNN-based approaches applied to the animal audio classification problem works better than the stand-alone CNNs.

show abstract

“…Over the last decade, research in sound classification and recognition has gained in popularity and rapidly broaden in its application from the more traditional focus on speech recognition [1] and music genre classification [2] to biometric identification [3], computer-aided heart sound detection [4], environmental audio scene and sound recognition [5,6], biodiversity assessment [7], human voice classification and emotion recognition [8], English accent classification and gender identification [9], to list a few of a widerange of application areas. As with research in pattern recognition generally, the features fed into classifiers were initially engineered, which in the case of sound applications meant extracting from raw audio traces such descriptors as the Statistical Spectrum Descriptor and Rhythm Histogram [10].…”

Section: Introductionmentioning

confidence: 99%

Animal Sound Classification Using Dissimilarity Spaces

Nanni¹,

Brahnam²,

Lumini³

et al. 2020

Preprint

View full text Add to dashboard Cite

The classifier system proposed in this work combines the dissimilarity spaces produced by a set of Siamese neural networks (SNNs) designed using 4 different backbones, with different clustering techniques for training SVMs for automated animal audio classification. The system is evaluated on two animal audio datasets: one for cat and another for bird vocalizations. Different clustering methods reduce the spectrograms in the dataset to a set of centroids that generate (in both a supervised and unsupervised fashion) the dissimilarity space through the Siamese networks. In addition to feeding the SNNs with spectrograms, additional experiments process the spectrograms using the Heterogeneous Auto-Similarities of Characteristics. Once the similarity spaces are computed, a vector space representation of each pattern is generated that is then trained on a Support Vector Machine (SVM) to classify a spectrogram by its dissimilarity vector. Results demonstrate that the proposed approach performs competitively (without ad-hoc optimization of the clustering methods) on both animal vocalization datasets. To further demonstrate the power of the proposed system, the best stand-alone approach is also evaluated on the challenging Dataset for Environmental Sound Classification (ESC50) dataset. The MATLAB code used in this study is available at https://github.com/LorisNanni.

show abstract

Spectrogram based multi-task audio classification

Cited by 135 publications

References 25 publications

A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition

A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition

Spectrogram Classification Using Dissimilarity Space

Animal Sound Classification Using Dissimilarity Spaces

Contact Info

Product

Resources

About