Masked Conditional Neural Networks for sound classification

Medhat, Fady; Chesmore, David; Robinson, John A.

doi:10.1016/j.asoc.2020.106073

Cited by 39 publications

(21 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…• An Ontology-Aware Framework for Audio Event Classification [65] • Masked Conditional Neural Networks for sound classification [43] • CascadeML: An Automatic Neural Network Architecture Evolution and Training Algorithm for Multi-label Classification [46] We now discuss how our work differs from these approaches.…”

Section: Cascading Architecturesmentioning

confidence: 99%

See 1 more Smart Citation

The AI Mechanic: Acoustic Vehicle Characterization Neural Networks

Terwilliger¹,

Siegel²

2022

Preprint

View full text Add to dashboard Cite

In a world increasingly dependent on road-based transportation, it is essential to understand vehicles. We introduce the AI mechanic, an acoustic vehicle characterization deep learning system, as an integrated approach using sound captured from mobile devices to enhance transparency and understanding of vehicles and their condition for non-expert users. We develop and implement novel cascading architectures for vehicle understanding, which we define as sequential, conditional, multilevel networks that process raw audio to extract highly-granular insights. To showcase the viability of cascading architectures, we build a multi-task convolutional neural network that predicts and cascades vehicle attributes to enhance fault detection. We train and test these models on a synthesized dataset reflecting more than 40 hours of augmented audio and achieve > 92% validation set accuracy on attributes (fuel type, engine configuration, cylinder count and aspiration type). Our cascading architecture additionally achieved 93.6% validation and 86.8% test set accuracy on misfire fault prediction, demonstrating margins of 16.4% / 7.8% and 4.2% / 1.5% improvement over naïve and parallel baselines. We explore experimental studies focused on acoustic features, data augmentation, feature fusion, and data reliability. Finally, we conclude with a discussion of broader implications, future directions, and application areas for this work.

show abstract

Section: Cascading Architecturesmentioning

confidence: 99%

“…Both [43] and our work utilize neural networks for acoustic classification. However, the conditional component of [43] does not correspond with multi-label dependence proposed in our cascading architecture but rather conditional dependence in the data itself.…”

Section: Cascading Architecturesmentioning

confidence: 99%

The AI Mechanic: Acoustic Vehicle Characterization Neural Networks

Terwilliger¹,

Siegel²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The highest accuracy is achieved by the model proposed in [44], which presents a CNN without max-pooling function and using Log-Mel audio feature extraction (CNN-Model-2 (No-maxpooling) + Log-Mel + augmentation). In [47], the application of the Masked Conditional Neural Networks for sound classification (MCLNN) to the problem of music genre classification, and ESR is presented. Both proposed models achieve high accuracy, but they are large models demanding prohibitive memory requirements for embedded platforms.…”

Section: Description Of the Modelmentioning

confidence: 99%

MosAIc: A Classical Machine Learning Multi-Classifier Based Approach against Deep Learning Classifiers for Embedded Sound Classification

et al. 2021

View full text Add to dashboard Cite

Environmental Sound Recognition has become a relevant application for smart cities. Such an application, however, demands the use of trained machine learning classifiers in order to categorize a limited set of audio categories. Although classical machine learning solutions have been proposed in the past, most of the latest solutions that have been proposed toward automated and accurate sound classification are based on a deep learning approach. Deep learning models tend to be large, which can be problematic when considering that sound classifiers often have to be embedded in resource constrained devices. In this paper, a classical machine learning based classifier called MosAIc, and a lighter Convolutional Neural Network model for environmental sound recognition, are proposed to directly compete in terms of accuracy with the latest deep learning solutions. Both approaches are evaluated in an embedded system in order to identify the key parameters when placing such applications on constrained devices. The experimental results show that classical machine learning classifiers can be combined to achieve similar results to deep learning models, and even outperform them in accuracy. The cost, however, is a larger classification time.

show abstract

“…Medhat et al [14] suggest a binary-masked CNN model that adopts a controlled systematic sparseness such as embedding a filterbank-like behavior within the network to preserve the spatial locality of features in the process of e experiments on GTZAN and UrbanSound8K have achieved the accuracies of 85.1% and 74.2%, respectively. In their model, they introduce a set of hidden layers in which each neuron establishes a connection with the input by the activation function through the influence of distinct active regions in the feature vector, so the spatial information of the learned feature is saved as these active weights' locations which are fixed in position.…”

Section: Related Workmentioning

confidence: 99%

Sound Classification Based on Multihead Attention and Support Vector Machine

Yang

Zhao

2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Sound classification is a broad area of research that has gained much attention in recent years. The sound classification systems based on recurrent neural networks (RNNs) and convolutional neural networks (CNNs) have undergone significant enhancements in the recognition capability of models. However, their computational complexity and inadequate exploration of global dependencies for long sequences restrict improvements in their classification results. In this paper, we show that there are still opportunities to improve the performance of sound classification by substituting the recurrent architecture with the parallel processing structure in the feature extraction. In light of the small-scale and high-dimension sound datasets, we propose the use of the multihead attention and support vector machine (SVM) for sound taxonomy. The multihead attention is taken as the feature extractor to obtain salient features, and SVM is taken as the classifier to recognize all categories. Extensive experiments are conducted across three acoustically characterized public datasets, UrbanSound8K, GTZAN, and IEMOCAP, by using two commonly used audio spectrograms as inputs, respectively, and we fully evaluate the impact of parameters and feature types on classification accuracy. Our results suggest that the proposed model can reach comparable performance with existing methods and reveal its strong generalization ability of sound taxonomy.

show abstract

Masked Conditional Neural Networks for sound classification

Cited by 39 publications

References 58 publications

The AI Mechanic: Acoustic Vehicle Characterization Neural Networks

The AI Mechanic: Acoustic Vehicle Characterization Neural Networks

MosAIc: A Classical Machine Learning Multi-Classifier Based Approach against Deep Learning Classifiers for Embedded Sound Classification

Sound Classification Based on Multihead Attention and Support Vector Machine

Contact Info

Product

Resources

About