Whistle detection and classification for whales based on convolutional neural networks

Jiajia, Jiang; Bu, Lingran; Duan, Fajie; Wang, Xianquan; Liu, Wei; Sun, Zhongbo; Li, Chunyue

doi:10.1016/j.apacoust.2019.02.007

Cited by 66 publications

(41 citation statements)

References 25 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cetaceans usually move in clusters with many individuals vocalizing simultaneously as they move [51]. The recorded sound is thus complex to analyze as a result of this simultaneous vocalization and the presence of anthropogenic noise [25], [51]. Estimating the abundance of species or types from the recorded sounds is often a problem because the sound could be from an individual vocalizing continuously or multiple individual vocalizing simultaneously.…”

Section: Volume 4 2016mentioning

confidence: 99%

“…The preprocessing stage focuses on recovering of frequency data and producing a time-frequency-amplitude representation of the recorded signal to form a dataset [25], [26]. This process include the denoising done to clean and enhance the quality of the whale sound [32].…”

Section: Volume 4 2016mentioning

confidence: 99%

“…Acoustic monitoring on the other hand, serves as an important avenue to monitor marine mammals at great distance (as far as 100km in some instances for low frequency calls) compared to visual methods, because sound can propagate much further in oceans than light [6], [13], [25]. Acoustic monitoring is also not affected by conditions under which visual method cannot perform.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Review of Automatic Detection and Classification Techniques for Cetacean Vocalization

2020

View full text Add to dashboard Cite

Cetaceans have elicited the attention of researchers in recent decades due to their importance to the ecosystem and their economic values. They use sound for communication, echolocation and other social activities. Their sounds are highly non-stationary, transitory and range from short to long sounds. Passive acoustic monitoring (PAM) is a popular method used for monitoring cetaceans in their ecosystems. The volumes of data accumulated using PAM are usually big, so they are difficult to analyze using manual inspection. Therefore different techniques with mixed outcomes have been developed for the automatic detection and classification of signals of different cetacean species. So far, no single technique developed is perfect to detect and classify the vocalizations of over 82 known species due to variability in time-frequency, difference in the amplitude among species and within species' vocal repertoire, physical environment, among others. The accuracy of any detector or classifier depends on the technique adopted as well as the nature of the signal to be analyzed. In this article, we review the existing techniques for the automatic detection and classification of cetacean vocalizations. We categorize the surveyed techniques, while emphasizing the advantages and disadvantages of these techniques. The article suggests possible research directions that can improve existing detection and classification techniques. In addition, the article recommends other suitable techniques that can be used to analyze non-linear and non-stationary signals such as the cetaceans' signals. Several research have been dedicated to this topic, however, there is no review of these past results that gives a quick overview in the area of cetacean detection and classification. This review will help researchers and practitioners in the field to make insightful decisions based on their requirements.

show abstract

Section: Volume 4 2016mentioning

confidence: 99%

Section: Volume 4 2016mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Review of Automatic Detection and Classification Techniques for Cetacean Vocalization

2020

View full text Add to dashboard Cite

show abstract

“…Case studies reporting successful applications play an important role in developing and disseminating best practices, and in discriminating between those tasks that current deep learning methods are able to automate and those they cannot. Previous applications have used convolutional neural networks (CNNs; LeCun, Bengio, and Hinton (2015)) to identify various bird (Grill & Schlüter, 2017;Kahl et al, 2017;Stowell, Wood, et al, 2019) and whale species (Bergler et al, 2019;Bermant, Bronstein, Wood, Gero, & Gruber, 2019;Jiang et al, 2019;Shiu et al, 2020), bees (Kulyukin, Mukherjee, & Amlathe, 2018;Nolasco et al, 2019), as well as anomalous acoustic events in soundscapes (Sethi et al, 2020). These have shown, for example, that a generally good approach is to represent data as spectrograms and treat the problem as an image classification one, as well as providing specialised approaches for data augmentation on spectrogram inputs, such as pitch and time shifting and introducing background noise (Bergler et al, 2019;Sprengel, Jaggi, Kilcher, & Hofmann, 2016).…”

Section: Introductionmentioning

confidence: 99%

Automated detection of Hainan gibbon calls for passive acoustic monitoring

Dufourq

Durbach

Hansford

et al. 2020

Preprint

View full text Add to dashboard Cite

1AbstractExtracting species calls from passive acoustic recordings is a common preliminary step to ecological analysis. For many species, particularly those occupying noisy, acoustically variable habitats, the call extraction process continues to be largely manual, a time-consuming and increasingly unsustainable process. Deep neural networks have been shown to offer excellent performance across a range of acoustic classification applications, but are relatively underused in ecology.We describe the steps involved in developing an automated classifier for a passive acoustic monitoring project, using the identification of calls of the Hainan gibbon (Nomascus hainanus), one of the world’s rarest mammal species, as a case study. This includes preprocessing - selecting a temporal resolution, windowing and annotation; data augmentation; processing - choosing and fitting appropriate neural network models; and postprocessing - linking model predictions to replace, or more likely facilitate, manual labelling.Our best model converted acoustic recordings into spectrogram images on the mel frequency scale, using these to train a convolutional neural network. Model predictions were highly accurate, with per-second false positive and false negative rates of 1.5% and 22.3%. Nearly all false negatives were at the fringes of calls, adjacent to segments where the call was correctly identified, so that very few calls were missed altogether. A postprocessing step identifying intervals of repeated calling reduced an eight-hour recording to, on average, 22 minutes for manual processing, and did not miss any calling bouts over 72 hours of test recordings. Gibbon calling bouts were detected regularly in multi-month recordings from all selected survey points within Bawangling National Nature Reserve, Hainan.We demonstrate that passive acoustic monitoring incorporating an automated classifier represents an effective tool for remote detection of one of the world’s rarest and most threatened species. Our study highlights the viability of using neural networks to automate or greatly assist the manual labelling of data collected by passive acoustic monitoring projects. We emphasise that model development and implementation be informed and guided by ecological objectives, and increase accessibility of these tools with a series of notebooks that allow users to build and deploy their own acoustic classifiers.

show abstract

“…Most methods for detecting and classifying clicks, whistles and pulsed calls from marine mammals have used standard audio analysis methods, such as the short-term Fourier transform [24], the wavelet transform [25], the Hilbert Huang transform [26], the Chirplet transform [27] and the Weyl transform [28]. The Hidden Markov Models (HMM) [29], the Support Vector Machine (SVM) [30] and the Deep Neural Network (DNN) models [31] were used to classify the audio features. Open source software for PAM of marine mammals is available, such as PAMGuard [32].…”

Section: Introductionmentioning

confidence: 99%

Spatial, Temporal and Spectral Multiresolution Analysis for the INTERSPEECH 2019 ComParE Challenge

Caraty¹,

Montacié²

2019

Interspeech 2019

View full text Add to dashboard Cite

The INTERSPEECH 2019 Orca Activity Challenge consists in the detection of the Orca sounds from underwater audio signal. Orca can produce a wide variety of sounds categorized in clicks, whistles and pulsed calls. Clicks are useful for echolocation, whistles and pulsed calls are used as social signals. Experiments were conducted on DeepAL Fieldwork Data (DLFD). Underwater sounds were recorded in northern British Columbia by a hydrophones array. Recordings were labeled by marine biologists in Orca sounds or Noise. We have investigated multiresolution analysis according to the three main relevant acoustic levels: spatial, temporal and spectral. For this purpose, we studied the beamforming array analysis, the multitemporal resolution and the multilevel wavelet decomposition. For the spatial level, a beamforming algorithm was used for denoising the underwater audio signal. For the temporal level, two sets of multitemporal three-level features were extracted using pyramidal representation. For the spectral level, in order to detect transient sound, wavelet analysis was computed using various wavelet families. At last, an Orca Activity detector was designed combining ComParE set with multitemporal and multilevel wavelet features. Experiments on the Test set have shown a significant improvement of 0.051, compared to the baseline performance of the Challenge (0.866).

show abstract

Whistle detection and classification for whales based on convolutional neural networks

Abstract: Jia-jia Jiang #, a, 1) , Ling-ran Bu #, a) , Fa-jie Duan a) , Xian-quan Wang a) , Wei Liu b) , Zhong-bo Sun a) and Chun-yue Li a)

Cited by 66 publications

References 25 publications

Review of Automatic Detection and Classification Techniques for Cetacean Vocalization

Review of Automatic Detection and Classification Techniques for Cetacean Vocalization

Automated detection of Hainan gibbon calls for passive acoustic monitoring

Spatial, Temporal and Spectral Multiresolution Analysis for the INTERSPEECH 2019 ComParE Challenge

Contact Info

Product

Resources

About