Convolutional recurrent neural networks for bird audio detection

Çakır, Emre; Adavanne, Sharath; Parascandolo, Giambattista; Drossos, Konstantinos; Virtanen, Tuomas

doi:10.23919/eusipco.2017.8081508

Cited by 80 publications

(47 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, the organizers of this challenge have managed to attract researchers from the machine learning and music information retrieval (MIR) communities [61]. This had led to the publication of new applications of existing machine learning methods to the domain of avian bioacoustics: these include multiple instance learning [62], convolutional recurrent neural networks [63], and densely connected convolutional networks [64]. Despite its undeniable merit of having gathered several data collection initiatives into a single cross-collection evaluation campaign, the methodology of the "bird detection in audio" challenge suffers from a lack of interpretability in the discussion of results post hoc.…”

Section: Evidence Of Technical Bias In State-of-the-art Bioacoustic Dmentioning

confidence: 99%

Robust sound event detection in bioacoustic sensor networks

et al. 2019

View full text Add to dashboard Cite

Bioacoustic sensors, sometimes known as autonomous recording units (ARUs), can record sounds of wildlife over long periods of time in scalable and minimally invasive ways. Deriving per-species abundance estimates from these sensors requires detection, classification, and quantification of animal vocalizations as individual acoustic events. Yet, variability in ambient noise, both over time and across sensors, hinders the reliability of current automated systems for sound event detection (SED), such as convolutional neural networks (CNN) in the time-frequency domain. In this article, we develop, benchmark, and combine several machine listening techniques to improve the generalizability of SED models across heterogeneous acoustic environments. As a case study, we consider the problem of detecting avian flight calls from a ten-hour recording of nocturnal bird migration, recorded by a network of six ARUs in the presence of heterogeneous background noise. Starting from a CNN yielding state-of-the-art accuracy on this task, we introduce two noise adaptation techniques, respectively integrating short-term (60 ms) and long-term (30 min) context. First, we apply per-channel energy normalization (PCEN) in the time-frequency domain, which applies short-term automatic gain control to every subband in the mel-frequency spectrogram. Secondly, we replace the last dense layer in the network by a context-adaptive neural network (CA-NN) layer, i.e. an affine layer whose weights are dynamically adapted at prediction time by an auxiliary network taking long-term summary statistics of spectrotemporal features as input. We show that PCEN reduces temporal overfitting across dawn vs. dusk audio clips whereas context adaptation on PCEN-based summary statistics reduces spatial overfitting across sensor locations. Moreover, combining them yields state-of-the-art results that are unmatched by artificial data augmentation alone. We release a pre-trained version of our best performing system under the name of BirdVoxDetect, a ready-to-use detector of avian flight calls in field recordings.

show abstract

Section: Evidence Of Technical Bias In State-of-the-art Bioacoustic Dmentioning

confidence: 99%

Robust sound event detection in bioacoustic sensor networks

et al. 2019

View full text Add to dashboard Cite

show abstract

“…4. The reason is, the sounds of birds are usually contained in a small portion of the frequency range (mostly around 2-8 kHz) as stated in [10], so we only extract features from the range of (0, 10) kHz. In order to focus only sounds pro- † http://www.xeno-canto.org duced in the vocal organ of birds (i.e.…”

Section: Databasementioning

confidence: 99%

“…Although sound is in some case complementary to visual information, such as when we listen to something out of view, vision and hearing are often informative about the same structures in the world [8]. As a consequence, numerous efforts have been devoted to recognize bird species based on auditory data [9], [10] in recent years. Adapting CNN architectures for the purpose of audio event detection has become a common practice and generating deep features based on visual representations of audio recordings has proven to be very effective [11] such as in bird sounds [10], [12].…”

Section: Introductionmentioning

confidence: 99%

“…As a consequence, numerous efforts have been devoted to recognize bird species based on auditory data [9], [10] in recent years. Adapting CNN architectures for the purpose of audio event detection has become a common practice and generating deep features based on visual representations of audio recordings has proven to be very effective [11] such as in bird sounds [10], [12]. To improve the affinity between images and sounds, we utilize the CNN to extract features from spectrogram of audio recordings.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cross-Domain Deep Feature Combination for Bird Species Classification with Audio-Visual Data

Bold

Zhang

Akashi

2019

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

In recent decade, many state-of-the-art algorithms on image classification as well as audio classification have achieved noticeable successes with the development of deep convolutional neural network (CNN). However, most of the works only exploit single type of training data. In this paper, we present a study on classifying bird species by exploiting the combination of both visual (images) and audio (sounds) data using CNN, which has been sparsely treated so far. Specifically, we propose CNN-based multimodal learning models in three types of fusion strategies (early, middle, late) to settle the issues of combining training data cross domains. The advantage of our proposed method lies on the fact that we can utilize CNN not only to extract features from image and audio data (spectrogram) but also to combine the features across modalities. In the experiment, we train and evaluate the network structure on a comprehensive CUB-200-2011 standard data set combing our originally collected audio data set with respect to the data species. We observe that a model which utilizes the combination of both data outperforms models trained with only an either type of data. We also show that transfer learning can significantly increase the classification performance.

show abstract

“…Contrary to the previous works, we postulate using recurrent convolutional neural networks as our feature extractor. The motivation to use this architecture stems from the fact that recurrent convolutional neural networks were successfully employed in similar acoustic modeling applications [27] and show state-of-the-art performances on many other related classification tasks, such as document [28], image [17], bird songs [18] and music genre classification [29]. Furthermore, we decided to use CQT-grams instead of raw audio signal, as an input to our recurrent convolutional neural network.…”

Section: Related Workmentioning

confidence: 99%

Speaker Diarization Using Deep Recurrent Convolutional Neural Networks for Speaker Embeddings

Cyrta¹,

Trzciński

Stokowiec

2017

Information Systems Architecture and Technology: Proceedings of 38th International Conference on Information Systems Architectu

View full text Add to dashboard Cite

In this paper we propose a new method of speaker diarization that employs a deep learning architecture to learn speaker embeddings. In contrast to the traditional approaches that build their speaker embeddings using manually hand-crafted spectral features, we propose to train for this purpose a recurrent convolutional neural network applied directly on magnitude spectrograms. To compare our approach with the state of the art, we collect and release for the public an additional dataset of over 6 hours of fully annotated broadcast material. The results of our evaluation on the new dataset and three other benchmark datasets show that our proposed method significantly outperforms the competitors and reduces diarization error rate by a large margin of over 30% with respect to the baseline.

show abstract

Convolutional recurrent neural networks for bird audio detection

Cited by 80 publications

References 13 publications

Robust sound event detection in bioacoustic sensor networks

Robust sound event detection in bioacoustic sensor networks

Cross-Domain Deep Feature Combination for Bird Species Classification with Audio-Visual Data

Speaker Diarization Using Deep Recurrent Convolutional Neural Networks for Speaker Embeddings

Contact Info

Product

Resources

About