Deep audio embeddings for vocalisation clustering

Best, Paul; Paris, Sébastien; Glotin, Hervé; Marxer, Ricard

doi:10.1101/2023.03.10.531956

Cited by 3 publications

(2 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Autoencoders have been used across a broad spectrum of research in various species to analyze acoustic communication [44,45,30,46,47]. The popularity of autoencoders is partly due to their inherent ability to learn latent data structures in an unsupervised fashion, i.e., without requiring annotated datasets.…”

Section: Discussionmentioning

confidence: 99%

SqueakOut: Autoencoder-based segmentation of mouse ultrasonic vocalizations

Santana,

Dietrich

2024

Preprint

View full text Add to dashboard Cite

Mice emit ultrasonic vocalizations (USVs) that are important for social communication. Despite great advancements in tools to detect USVs from audio files in recent years, highly accurate segmentation of USVs from spectrograms (i.e., removing noise) remains a significant challenge. Here, we present a new dataset of 12,954 annotated spectrograms explicitly labeled for mouse USV segmentation. Leveraging this dataset, we developed SqueakOut, a lightweight (4.6M parameters) fully convolutional autoencoder that achieves high accuracy in supervised segmentation of USVs from spectrograms, with a Dice score of 90.22. SqueakOut combines a MobileNetV2 backbone with skip connections and transposed convolutions to precisely segment USVs. Using stochastic data augmentation techniques and a hybrid loss function, SqueakOut learns robust segmentation across varying recording conditions. We evaluate SqueakOut's performance, demonstrating substantial improvements over existing methods like VocalMat (63.82 Dice score). The accurate USV segmentations enabled by SqueakOut will facilitate novel methods for vocalization classification and more accurate analysis of mouse communication. To promote further research, we release the annotated 12,954 spectrogram USV segmentation dataset and the SqueakOut implementation publicly.

show abstract

Section: Discussionmentioning

confidence: 99%

SqueakOut: Autoencoder-based segmentation of mouse ultrasonic vocalizations

Santana,

Dietrich

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…HDBSCAN has the particularity of allowing soft clustering, where each vocal unit is not assigned to a single cluster, but instead to all clusters with varying probabilities. Both UMAP and HDBSCAN have become state-of-the-art algorithms due to their performance and robustness [69,70].…”

Section: Methodsmentioning

confidence: 99%

Vocal complexity in a socially complex corvid: gradation, diversity and lack of common call repertoire in male rooks

Martin,

Cornero,

Clayton

et al. 2024

R. Soc. Open Sci.

View full text Add to dashboard Cite

Vocal communication is widespread in animals, with vocal repertoires of varying complexity. The social complexity hypothesis predicts that species may need high vocal complexity to deal with complex social organization (e.g. have a variety of different interindividual relations). We quantified the vocal complexity of two geographically distant captive colonies of rooks, a corvid species with complex social organization and cognitive performances, but understudied vocal abilities. We quantified the diversity and gradation of their repertoire, as well as the inter-individual similarity at the vocal unit level. We found that males produced call units with lower diversity and gradation than females, while song units did not differ between sexes. Surprisingly, while females produced highly similar call repertoires, even between colonies, each individual male produced almost completely different call repertoires from any other individual. These findings question the way male rooks communicate with their social partners. We suggest that each male may actively seek to remain vocally distinct, which could be an asset in their frequently changing social environment. We conclude that inter-individual similarity, an understudied aspect of vocal repertoires, should also be considered as a measure of vocal complexity.

show abstract

Soundscape Characterization Using Autoencoders and Unsupervised Learning

Nieto-Mora,

Ferreira de Oliveira,

Sanchez-Giraldo

et al. 2024

Sensors

View full text Add to dashboard Cite

Passive acoustic monitoring (PAM) through acoustic recorder units (ARUs) shows promise in detecting early landscape changes linked to functional and structural patterns, including species richness, acoustic diversity, community interactions, and human-induced threats. However, current approaches primarily rely on supervised methods, which require prior knowledge of collected datasets. This reliance poses challenges due to the large volumes of ARU data. In this work, we propose a non-supervised framework using autoencoders to extract soundscape features. We applied this framework to a dataset from Colombian landscapes captured by 31 audiomoth recorders. Our method generates clusters based on autoencoder features and represents cluster information with prototype spectrograms using centroid features and the decoder part of the neural network. Our analysis provides valuable insights into the distribution and temporal patterns of various sound compositions within the study area. By utilizing autoencoders, we identify significant soundscape patterns characterized by recurring and intense sound types across multiple frequency ranges. This comprehensive understanding of the study area’s soundscape allows us to pinpoint crucial sound sources and gain deeper insights into its acoustic environment. Our results encourage further exploration of unsupervised algorithms in soundscape analysis as a promising alternative path for understanding and monitoring environmental changes.

show abstract

Deep audio embeddings for vocalisation clustering

Cited by 3 publications

References 53 publications

SqueakOut: Autoencoder-based segmentation of mouse ultrasonic vocalizations

SqueakOut: Autoencoder-based segmentation of mouse ultrasonic vocalizations

Vocal complexity in a socially complex corvid: gradation, diversity and lack of common call repertoire in male rooks

Soundscape Characterization Using Autoencoders and Unsupervised Learning

Contact Info

Product

Resources

About