Classifying sex and strain from mouse ultrasonic vocalizations using deep learning

Ivanenko, Aleksandr; Watkins, Paul V.; Gerven, Marcel A. J. van; Hammerschmidt, Kurt; Englitz, Bernhard

doi:10.1371/journal.pcbi.1007918

Cited by 35 publications

(48 citation statements)

References 43 publications

(80 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, we note that almost all recordings of the same individuals are co-localized, indicating that within-subject differences in syllable repertoire are smaller than those between individuals. Although it has been previously shown that a deep convolutional neural network can be trained to classify USV syllables according to mouse identity with good accuracy ( Ivanenko et al, 2020 , Figure S1), here we find that repertoire features learned in a wholly unsupervised fashion achieve similar results, indicating that mice produce individually stereotyped, stable vocal repertoires.…”

Section: Resultssupporting

confidence: 45%

“…Finally, we used a much larger library of female-directed mouse USVs (36 individuals, 2-4 20minute recording sessions each, 40 total hours of audio, 156,000 syllables) to investigate the diversity and stability of syllable repertoires. We repeated the above procedure, estimating MMD for shown that a deep convolutional neural network can be trained to classify USV syllables according to mouse identity with good accuracy (Ivanenko et al (2020), Figure S1), here we find that repertoire features learned in a wholly unsupervised fashion achieve similar results, indicating mice produce individually-stereotyped, stable vocal repertoires.…”

Section: Latent Spaces Facilitate Comparisons Between Vocal Repertoiresmentioning

confidence: 80%

See 1 more Smart Citation

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Goffinet

Brudner

Mooney

et al. 2021

eLife

118

View full text Add to dashboard Cite

Increases in the scale and complexity of behavioral data pose an increasing challenge for data analysis. A common strategy involves replacing entire behaviors with small numbers of handpicked, domain-specific features, but this approach suffers from several crucial limitations. For example, handpicked features may miss important dimensions of variability, and correlations among them complicate statistical testing. Here, by contrast, we apply the variational autoencoder (VAE), an unsupervised learning method, to learn features directly from data and quantify the vocal behavior of two model species: the laboratory mouse and the zebra finch. The VAE converges on a parsimonious representation that outperforms handpicked features on a variety of common analysis tasks, enables the measurement of moment-by-moment vocal variability on the timescale of tens of milliseconds in the zebra finch, provides strong evidence that mouse ultrasonic vocalizations do not cluster as is commonly believed, and captures the similarity of tutor and pupil birdsong with qualitatively higher fidelity than previous approaches. In all, we demonstrate the utility of modern unsupervised learning approaches to the quantification of complex and high-dimensional vocal behavior.

show abstract

Section: Resultssupporting

confidence: 45%

Section: Latent Spaces Facilitate Comparisons Between Vocal Repertoiresmentioning

confidence: 80%

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Goffinet

Brudner

Mooney

et al. 2021

eLife

118

View full text Add to dashboard Cite

show abstract

“…Furthermore, we note that almost all recordings of the same individuals are co-localized, indicating that within-subject differences in syllable repertoire are smaller than those between individuals. Although it has been previously shown that a deep convolutional neural network can be trained to classify USV syllables according to mouse identity with good accuracy ([19], Figure S1), here we find that repertoire features learned in a wholly unsupervised fashion achieve similar results, indicating mice produce individually-stereotyped, stable vocal repertoires.…”

Section: Resultssupporting

confidence: 40%

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Goffinet

Brudner

Mooney

2019

Preprint

View full text Add to dashboard Cite

Vocalization is an essential medium for social and sexual signaling in most birds and mammals. 1 Consequently, the analysis of vocal behavior is of great interest to fields such as neuroscience and 2 linguistics. A standard approach to analyzing vocalization involves segmenting the sound stream 3 into discrete vocal elements, calculating a number of handpicked acoustic features, and then using 4 the feature values for subsequent quantitative analysis. While this approach has proven powerful, 5 it suffers from several crucial limitations: First, handpicked acoustic features may miss important 6 dimensions of variability that are important for communicative function. Second, many analyses 7 assume vocalizations fall into discrete vocal categories, often without rigorous justification. Third, a 8 syllable-level analysis requires a consistent definition of syllable boundaries, which is often difficult 9 to maintain in practice and limits the sorts of structure one can find in the data. To address these 10 shortcomings, we apply a data-driven approach based on the variational autoencoder (VAE), an 11 unsupervised learning method, to the task of characterizing vocalizations in two model species: 12 the laboratory mouse (Mus musculus) and the zebra finch (Taeniopygia guttata). We find that the 13 VAE converges on a parsimonious representation of vocal behavior that outperforms handpicked 14 acoustic features on a variety of common analysis tasks, including representing acoustic similarity and 15 recovering a known effect of social context on birdsong. Additionally, we use our learned acoustic 16 features to argue against the widespread view that mouse ultrasonic vocalizations form discrete 17 syllable categories. Lastly, we present a novel "shotgun VAE" that can quantify moment-by-moment 18 variability in vocalizations. In all, we show that data-derived acoustic features confirm and extend 19 existing approaches while offering distinct advantages in several critical applications. 20 1 Introduction 21 Vocalization is an essential medium for social and sexual signaling in most birds and mammals, and also serves as a 22 natural substrate for language and music in humans. Consequently, the analysis of vocal behavior is of great interest to 23 ethologists, psychologists, linguists, and neuroscientists. A major goal of these various lines of enquiry is to develop 24 methods for quantitative analysis of vocal behavior, efforts that have resulted in several powerful methods that enable 25 the automatic or semi-automatic analysis of vocalizations. Key to this approach has been the existence of software 26 packages that calculate acoustic features for each syllable within a vocalization [4, 39, 40, 7, 6]. For example, Sound 27 Analysis Pro, focused on birdsong, calculates 14 features for each syllable, including duration, spectral entropy, and 28 goodness of pitch, and uses these as a basis for subsequent clustering and analysis [39]. More recently, MUPET and 29 DeepSqueak have applied a similar approach to mouse vocalizati...

show abstract

“…Mice produce ultrasonic vocalizations (USVs) in diverse social contexts ranging from courtship to aggression ( Sangiamo et al, 2020 ; Warren et al, 2020 ; Neunuebel et al, 2015 ). We tested DAS using audio from an intruder assay, in which an anesthetized female was put into the home cage and the USVs produced by a resident female or male were recorded ( Ivanenko et al, 2020 ). The female USVs from this assay typically consist of pure tones with weak harmonics and smooth frequency modulations that are often interrupted by frequency steps ( Figure 2A, B ).…”

Section: Resultsmentioning

confidence: 99%

Fast and accurate annotation of acoustic signals with deep neural networks

Steinfath

Palacios

Rottschäfer

et al. 2021

Preprint

View full text Add to dashboard Cite

Acoustic signals serve communication within and across species throughout the animal kingdom. Studying the genetics, evolution, and neurobiology of acoustic communication requires annotating acoustic signals: segmenting and identifying individual acoustic elements like syllables or sound pulses. To be useful, annotations need to be accurate, robust to noise, fast. We introduce DeepSS, a method that annotates acoustic signals across species based on a deep-learning derived hierarchical presentation of sound. We demonstrate the accuracy, robustness, and speed of DeepSS using acoustic signals with diverse characteristics: courtship song from flies, ultrasonic vocalizations of mice, and syllables with complex spectrotemporal structure from birds. DeepSS comes with a graphical user interface for annotating song, training the network, and for generating and proofreading annotations (available at https://janclemenslab.org/deepss). The method can be trained to annotate signals from new species with little manual annotation and can be combined with unsupervised methods to discover novel signal types. DeepSS annotates song with high throughput and low latency, allowing realtime annotations for closed-loop experimental interventions.

show abstract

Classifying sex and strain from mouse ultrasonic vocalizations using deep learning

Cited by 35 publications

References 43 publications

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Fast and accurate annotation of acoustic signals with deep neural networks

Contact Info

Product

Resources

About