Recurrent Convolutional Neural Networks for Large Scale Bird Species Classification

Abstract

We present a deep learning approach towards the large-scale prediction and analysis of bird acoustics from 100 different bird species. We use spectrograms constructed on bird audio recordings from the Cornell Bird Challenge (CBC) dataset, which includes recordings with background noise, of multiple and potentially overlapping bird vocalizations per audio. Our experiments show that a hybrid modeling approach that involves a Convolutional Neural Network (CNN) for learning the representation for a slice of the spectrogram and a Recurrent Neural Network (RNN) for the temporal component to combine across time-points leads to the most accurate model on this dataset. We show results on a spectrum of models ranging from stand-alone CNNs to hybrid models of various types obtained by combining CNNs with CNNs or RNNs of the following types: Long Short-Term Memory (LSTM) networks, Gated Recurrent Units (GRU) and Legendre Memory Units (LMU). The best performing model achieves an average accuracy of 67% over the 100 different bird species, with the highest accuracy of 90% for the Red crossbill. We further analyze the learned representations visually and find them to be intuitive, where we find that related bird species are clustered close together. We present a novel way to empirically interpret the representations learned by the LMU-based hybrid model which shows how memory channel patterns over time change with spectrograms.

Literature Survey on Bird Call Identification and Monitoring

“…The research in [1] introduces a deep learning approach for predicting and analyzing bird acoustics using a dataset containing recordings of 100 bird species. The study explores various neural network architectures, including hybrid models that combine CNNs with other CNNs or RNNs such as LSTM, GRU, and LMU.…”

“…The best-performing model achieves noteworthy accuracy across 100 bird species, showcasing the potential of the approach. [1] Studies in [2] address a critical need in environmental monitoring by proposing an innovative approach for automated bird call recognition using ResNet-50, an architecture known for its success in computer vision tasks. The authors, Dmitry Konovalov et.…”

“…Gaurav Gupta and colleagues [1] utilized the Cornell Bird Challenge (CBC) 2020 dataset, comprising 264 bird species, each accompanied by varying audio samples ranging from 9 to 1778 for every species. This dataset contains recordings capturing multiple and potentially overlapping bird vocalizations amidst background noise, including up to 100 different bird species.…”

“…The hybrid:mode of audio signal processing described in [1] utilizes the sliding window mechanism, which is specifically designed for bird sound classification. This mechanism traverses raw audio clips with a sliding window of a specified length (Ws) and hop length (Hs), thereby capturing temporal components and generating 26 slides from a 7-second audio clip.…”

“…The resulting output from the block is directed to the final classification block, which produces softmax outputs. The schematic representation of this hybrid methodology is illustrated in Figure 1 [1], providing a detailed visualization of the model's architecture. This hybrid approach is highly effective in analyzing the temporal aspects of audio signals and has been praised by researchers in [1] for its efficiency in bird sound classification.…”