A hybrid technique for speech segregation and classification using a sophisticated deep neural network

Qazi, Khurram Ashfaq; Nawaz, Tabassam; Mehmood, Zahid; Rashid, Muhammad; Habib, Hafiz Adnan

doi:10.1371/journal.pone.0194151

Cited by 28 publications

(18 citation statements)

References 30 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All audio was converted into mono, as done in various applications (e.g. Bergler et al, 2019;Qazi, Tabassam Nawaz, Rashid, & Habib, 2018;Stowell, Petrusková, Šálek, & Linhart, 2019). By crossreferencing the time intervals of each segment with the logged start and end times of known gibbon phrases, each segment was labelled as (a) a "presence", if its time interval completely contained the interval of at least one labelled phrase, (b) an "absence", if its time interval contained no part of any phrase, or (c) a "partial presence", if its time interval intersected but did not completely contain the interval of at least one labelled phrase ( Figure 2).…”

Section: Discussionmentioning

confidence: 99%

Automated detection of Hainan gibbon calls for passive acoustic monitoring

Dufourq

Durbach

Hansford

et al. 2020

Preprint

View full text Add to dashboard Cite

1AbstractExtracting species calls from passive acoustic recordings is a common preliminary step to ecological analysis. For many species, particularly those occupying noisy, acoustically variable habitats, the call extraction process continues to be largely manual, a time-consuming and increasingly unsustainable process. Deep neural networks have been shown to offer excellent performance across a range of acoustic classification applications, but are relatively underused in ecology.We describe the steps involved in developing an automated classifier for a passive acoustic monitoring project, using the identification of calls of the Hainan gibbon (Nomascus hainanus), one of the world’s rarest mammal species, as a case study. This includes preprocessing - selecting a temporal resolution, windowing and annotation; data augmentation; processing - choosing and fitting appropriate neural network models; and postprocessing - linking model predictions to replace, or more likely facilitate, manual labelling.Our best model converted acoustic recordings into spectrogram images on the mel frequency scale, using these to train a convolutional neural network. Model predictions were highly accurate, with per-second false positive and false negative rates of 1.5% and 22.3%. Nearly all false negatives were at the fringes of calls, adjacent to segments where the call was correctly identified, so that very few calls were missed altogether. A postprocessing step identifying intervals of repeated calling reduced an eight-hour recording to, on average, 22 minutes for manual processing, and did not miss any calling bouts over 72 hours of test recordings. Gibbon calling bouts were detected regularly in multi-month recordings from all selected survey points within Bawangling National Nature Reserve, Hainan.We demonstrate that passive acoustic monitoring incorporating an automated classifier represents an effective tool for remote detection of one of the world’s rarest and most threatened species. Our study highlights the viability of using neural networks to automate or greatly assist the manual labelling of data collected by passive acoustic monitoring projects. We emphasise that model development and implementation be informed and guided by ecological objectives, and increase accessibility of these tools with a series of notebooks that allow users to build and deploy their own acoustic classifiers.

show abstract

Section: Discussionmentioning

confidence: 99%

Automated detection of Hainan gibbon calls for passive acoustic monitoring

Dufourq

Durbach

Hansford

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…CNNs are variant of deep learning and substitute for auditory representations. Commonly combined with other machine learning techniques to derive DNNs for voice separation and identification [18,19]. The DNN and CNN are very similar but the difference between them is that CNN has additional features extracting layers.…”

Section: A Cnn Modelmentioning

confidence: 99%

“…These additional features extracting layers are used to generate input descriptions for sub-sequent levels to the DNNs in place of initially processed features. Each one of the input features consists of a part of a larger part of convolution and max-pooling units [19,20]. The following explains the basic concepts of CNN architecture: Convolutional layer.…”

Section: A Cnn Modelmentioning

confidence: 99%

Speech Segregation in Background Noise Based on Deep Learning

et al. 2020

View full text Add to dashboard Cite

“…The image features obtained by this method are capable of producing good retrieval performance and have the advantages of fast indexing and scalability. Z. Mehmood et al [27] proposed a new method for speech separation of unmarked static noise audio signals using the deep trusted network (DBN) model. The method can effectively separate the music signal from the noisy audio stream, remove the static noise using the hidden layer separation model of the recurrent neural network (RNN), and then use the dictionary-based Fisher algorithm for speech classification.…”

Section: Related Workmentioning

confidence: 99%

Multilayer Convolutional Feature Aggregation Algorithm for Image Retrieval

Dong

Liu

2019

Mathematical Problems in Engineering

View full text Add to dashboard Cite

In image retrieval tasks, the single-layer convolutional feature has insufficient image semantic representation ability. A new image description algorithm ML-RCroW based on multilayer multiregion cross-weighted aggregational deep convolutional features is proposed. First, the ML-RCroW algorithm inputs an image into the VGG16 (a deep convolutional neural network developed by researchers at Visual Geometry Group and Google DeepMind) network model in which the fully connected layer is discarded. The visual feature information in the convolutional neural network (CNN) is extracted, and the target response weight map is generated by combining with the spatial weighting algorithm of the target fuzzy marker. Then, visual features in the CNN are divided into multiple regions, and the pixels of each region are weighted by regional spatial weight, regional channel weight, and regional weight. The image global vector is generated by aggregating and encoding every region in the weighted feature map. Finally, features of each layer of the VGG16 network model are extracted and then aggregated and dimensionally reduced to obtain the final feature vector of the image. The experiments are carried out on the Oxford5k and Paris6k datasets provided by Oxford VGG. The experimental results show that the average accuracy of image retrieval based on the image feature description algorithm ML-RCroW is better than that achieved by the other commonly used algorithms such as SPoC, R-MAC, and CroW.

show abstract

A hybrid technique for speech segregation and classification using a sophisticated deep neural network

Cited by 28 publications

References 30 publications

Automated detection of Hainan gibbon calls for passive acoustic monitoring

Automated detection of Hainan gibbon calls for passive acoustic monitoring

Speech Segregation in Background Noise Based on Deep Learning

Multilayer Convolutional Feature Aggregation Algorithm for Image Retrieval

Contact Info

Product

Resources

About