High order structure in mouse courtship vocalizations

Hertz, Stav; Weiner, Benjamin; Perets, Nisim; London, Michael

doi:10.1101/728477

Cited by 3 publications

(7 citation statements)

References 29 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In species such as the Bengalese finch, where syllables are highly stereotyped, clustering syllables into discrete categories is a natural way to abstract song. The utility of clustering song elements in other species, however, is more contentious because discrete category boundaries are not as easily discerned [ 10 , 11 , 35 , 57 ].…”

Section: Resultsmentioning

confidence: 99%

“…Fig 10 ), but may disagree for a sizable minority of syllables ( Table 1 ). Similarly, in mice, different algorithmic methods for abstracting and transcribing mouse vocal units (USVs) can result in substantial differences between syntactic descriptions of sequential organization [ 57 ]. We were interested in the differences between the abstracted sequential organization of birdsong when syllables were labeled by hand versus clustered in latent space.…”

Section: Resultsmentioning

confidence: 99%

“…As was observed on the basis of the Hopkin’s statistic ( Fig 8 ), however, USVs do not cluster into discrete distributions in the same manner as birdsong. Choosing different arbitrary clustering heuristics will therefore have profound impacts on downstream analyses of sequential organization [ 57 ].…”

Section: Resultsmentioning

confidence: 99%

“…Similarly, latent models are playing an increasingly important role in computational ethology [ 17 ], where characterizations of animal movements and behaviors have uncovered complex sequential organization [ 38 , 79 , 80 ]. In animal communication, pattern recognition using various machine learning techniques has been used to characterize vocalizations and label auditory objects [ 3 , 34 , 35 , 39 , 57 , 77 , 78 ]. Our work furthers this emerging research area by demonstrating the utility of unsupervised latent models for both systematically visualizing and abstracting structure from animal vocalizations across a wide range of species.…”

Section: Discussionmentioning

confidence: 99%

See 3 more Smart Citations

Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires

2020

View full text Add to dashboard Cite

Animals produce vocalizations that range in complexity from a single repeated call to hundreds of unique vocal elements patterned in sequences unfolding over hours. Characterizing complex vocalizations can require considerable effort and a deep intuition about each species’ vocal behavior. Even with a great deal of experience, human characterizations of animal communication can be affected by human perceptual biases. We present a set of computational methods for projecting animal vocalizations into low dimensional latent representational spaces that are directly learned from the spectrograms of vocal signals. We apply these methods to diverse datasets from over 20 species, including humans, bats, songbirds, mice, cetaceans, and nonhuman primates. Latent projections uncover complex features of data in visually intuitive and quantifiable ways, enabling high-powered comparative analyses of vocal acoustics. We introduce methods for analyzing vocalizations as both discrete sequences and as continuous latent variables. Each method can be used to disentangle complex spectro-temporal structure and observe long-timescale organization in communication.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires

2020

View full text Add to dashboard Cite

show abstract

“…However, as seen above, mouse USVs clump together in a single quasi-continuous mass (Figure 5b). This raises a puzzle, since the clustering of mouse vocalizations is often considered well-established in the literature [17, 4, 48, 6, 15] and is assumed in most other analyses of these data [45, 7]. Clusters of mouse USVs are used to assess differences across strains [45], social contexts [6, 7, 14], and genotypes [12], and the study of transition models among clusters of syllables has given rise to models of syllable sequences that do not readily extend to the non-clustered case [17, 6].…”

Section: Resultsmentioning

confidence: 99%

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Goffinet

Brudner

Mooney

2019

Preprint

View full text Add to dashboard Cite

Vocalization is an essential medium for social and sexual signaling in most birds and mammals. 1 Consequently, the analysis of vocal behavior is of great interest to fields such as neuroscience and 2 linguistics. A standard approach to analyzing vocalization involves segmenting the sound stream 3 into discrete vocal elements, calculating a number of handpicked acoustic features, and then using 4 the feature values for subsequent quantitative analysis. While this approach has proven powerful, 5 it suffers from several crucial limitations: First, handpicked acoustic features may miss important 6 dimensions of variability that are important for communicative function. Second, many analyses 7 assume vocalizations fall into discrete vocal categories, often without rigorous justification. Third, a 8 syllable-level analysis requires a consistent definition of syllable boundaries, which is often difficult 9 to maintain in practice and limits the sorts of structure one can find in the data. To address these 10 shortcomings, we apply a data-driven approach based on the variational autoencoder (VAE), an 11 unsupervised learning method, to the task of characterizing vocalizations in two model species: 12 the laboratory mouse (Mus musculus) and the zebra finch (Taeniopygia guttata). We find that the 13 VAE converges on a parsimonious representation of vocal behavior that outperforms handpicked 14 acoustic features on a variety of common analysis tasks, including representing acoustic similarity and 15 recovering a known effect of social context on birdsong. Additionally, we use our learned acoustic 16 features to argue against the widespread view that mouse ultrasonic vocalizations form discrete 17 syllable categories. Lastly, we present a novel "shotgun VAE" that can quantify moment-by-moment 18 variability in vocalizations. In all, we show that data-derived acoustic features confirm and extend 19 existing approaches while offering distinct advantages in several critical applications. 20 1 Introduction 21 Vocalization is an essential medium for social and sexual signaling in most birds and mammals, and also serves as a 22 natural substrate for language and music in humans. Consequently, the analysis of vocal behavior is of great interest to 23 ethologists, psychologists, linguists, and neuroscientists. A major goal of these various lines of enquiry is to develop 24 methods for quantitative analysis of vocal behavior, efforts that have resulted in several powerful methods that enable 25 the automatic or semi-automatic analysis of vocalizations. Key to this approach has been the existence of software 26 packages that calculate acoustic features for each syllable within a vocalization [4, 39, 40, 7, 6]. For example, Sound 27 Analysis Pro, focused on birdsong, calculates 14 features for each syllable, including duration, spectral entropy, and 28 goodness of pitch, and uses these as a basis for subsequent clustering and analysis [39]. More recently, MUPET and 29 DeepSqueak have applied a similar approach to mouse vocalizati...

show abstract

Latent space visualization, characterization, and generation of diverse vocal communication signals

Sainburg

Thielk

Gentner

2019

Preprint

View full text Add to dashboard Cite

Animals produce vocalizations that range in complexity from a single repeated call to hundreds 1 of unique vocal elements patterned in sequences unfolding over hours. Characterizing complex 2 vocalizations can require considerable effort and a deep intuition about each species' vocal behavior. 3 Even with a great deal of experience, human characterizations of animal communication can be 4 affected by human perceptual biases. We present here a set of computational methods that center 5 around projecting animal vocalizations into low dimensional latent representational spaces that 6 are directly learned from data. We apply these methods to diverse datasets from over 20 species, 7 including humans, bats, songbirds, mice, cetaceans, and nonhuman primates, enabling high-powered 8 comparative analyses of unbiased acoustic features in the communicative repertoires across species. 9 Latent projections uncover complex features of data in visually intuitive and quantifiable ways. We 10 introduce methods for analyzing vocalizations as both discrete sequences and as continuous latent 11 variables. Each method can be used to disentangle complex spectro-temporal structure and observe 12 long-timescale organization in communication. Finally, we show how systematic sampling from 13 latent representational spaces of vocalizations enables comprehensive investigations of perceptual 14 and neural representations of complex and ecologically relevant acoustic feature spaces. 15Of the thousands of species that communicate vocally, the repertoires of only a tiny minority have been characterized or 18 studied in detail. This is due, in large part, to traditional analysis methods that require a high level of expertise that is 19 hard to develop and often species-specific. Here, we present a set of novel methods to project animal vocalizations 20 into latent feature spaces to quantitatively compare and develop visual intuitions about animal vocalizations, and to 21 systematically synthesize novel species-typical vocalizations from learned feature sets. We demonstrate these methods 22 across a series of analyses over 19 datasets of animal vocalizations from 29 different species, including songbirds, mice, 23 monkeys, humans, and whales. We show how learned latent feature spaces untangle complex spectro-temporal structure, 24 enable unbiased comparisons, and uncover high-level features such as individual identity and population dialects. We 25 generate smoothly varying morphs between vocalizations from a songbird species with a spectro-temporally complex 26 vocal repertoire, European starlings, and show how these methods enable a new degree of control over ecologically 27 relevant signals that can be broadly applied across behavioral and physiological experimental settings. 28 2 Introduction 29Vocal communication is a social behavior common to much of the animal kingdom in which acoustic signals are 30 transmitted from sender to receiver to convey various forms of information such as identity, individual fitness, or the 31 presence ...

show abstract

High order structure in mouse courtship vocalizations

Cited by 3 publications

References 29 publications

Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires

Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Latent space visualization, characterization, and generation of diverse vocal communication signals

Contact Info

Product

Resources

About