Reconstruction of physiological instructions from Zebra finch song

Perl, Yonatan Sanz; Arneodo, Ezequiel M.; Amador, Ana; Goller, Franz; Mindlin, Gabriel B.

doi:10.1103/physreve.84.051909

Cited by 44 publications

(76 citation statements)

References 27 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Biomechanical models describing these oscillations using two time dependent parameters (the tension in a ventral syringeal muscle (vS) and the subsyringeal air pressure) allowed to reproduce realistic synthesis of song. [16][17][18] These variables can be measured experimentally. In this work, we a)…”

Section: Pressure Gesturesmentioning

confidence: 99%

Evidence and control of bifurcations in a respiratory system

Goldin

Mindlin

2013

Chaos: An Interdisciplinary Journal of Nonlinear Science

Self Cite

View full text Add to dashboard Cite

We studied the pressure patterns used by domestic canaries in the production of birdsong. Acoustically different sound elements ("syllables") were generated by qualitatively different pressure gestures. We found that some ubiquitous transitions between syllables can be interpreted as bifurcations of a low dimensional dynamical system. We interpreted these results as evidence supporting a model in which different timescales interact nonlinearly. V C 2013 AIP Publishing LLC.[http://dx.doi.org/10.1063/1.4854395]The comprehension of motor activities that give rise to complex behavior are of great interest. A widely used animal model to study them is the one of singing birds, where respiration plays a fundamental role. Different mechanisms have been proposed to account for the dynamical origin of these motor gestures. In this work, we study the temporal signals of the pressure gestures used by domestic canaries (Serinus canaria) to perform their songs. Their analysis revealed that ubiquitous occurring transitions between syllables can be interpreted as the bifurcations of a low dimensional dynamical system. This provides evidence supporting a model where different nonlinearly interacting timescales participate.

show abstract

Section: Pressure Gesturesmentioning

confidence: 99%

Evidence and control of bifurcations in a respiratory system

Goldin

Mindlin

2013

Chaos: An Interdisciplinary Journal of Nonlinear Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…First, birdsong is temporally structured (like human speech); this temporal patterning can be built into a decoder using a recurrent neural network 5 . Second, the biomechanics of birdsong production are well understood; this enables us to employ a biophysical model of the vocal organ that captures most of the complexity of the song and reduces it to a lower dimensional parameter space 6,7 . By combining these techniques, we decode realistic synthetic birdsong directly from neural activity.…”

mentioning

confidence: 99%

“…1b). The dynamics of these labia can be modeled after the motion equations of a nonlinear oscillator, in which the features of the sounds produced are determined by only two parameters 6,16,17 , which represent the physiological motor instructions driving the syrinx (the respiratory and syringeal muscular activities) 6 . This model can produce synthetic vocalizations in hard real time 7 , and vocalizations generated by it are realistic, in the sense that their playback to the asleep/anesthetized animal elicits auditory responses in HVC neurons that match the high selectivity observed in response to BOS 9 .…”

mentioning

confidence: 99%

“…Syringeal and respiratory muscles act coordinately to modulate the flow of air through sets of labia and produce sound 15 . The complex labial motion is captured by the equations of a nonlinear oscillator 17 ; parameters that define acoustic properties of the sounds are surrogates of the activities of syringeal and respiratory muscles 6 . (c) To reproduce a particular vocalization (top) from the biophysical model, we fit the parameters (middle {α(t), β(t), e(t)}) such that upon integration, the synthetic song (bottom) matches the pitch and spectral richness (see Methods).…”

mentioning

confidence: 99%

See 1 more Smart Citation

A neural decoder for learned vocal behavior

Arneodo

Chen

Gilja

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

State of the art BMIs succeed at decoding behavioral intention from brain activity by mapping features of neuronal ensemble activity onto a motor space 1,2 . Yet the these motor spaces are confined by current technologies to rather simple actions. To prototype a decoder of complex, natural communication signals from neural activity, we capitalize on two aspects of birdsong, a powerful animal model for vocal learning that shares many features with humans speech 3,4 . First, birdsong is temporally structured (like human speech); this temporal patterning can be built into a decoder using a recurrent neural network 5 . Second, the biomechanics of birdsong production are well understood; this enables us to employ a biophysical model of the vocal organ that captures most of the complexity of the song and reduces it to a lower dimensional parameter space 6,7 . By combining these techniques, we decode realistic synthetic birdsong directly from neural activity.Our decoder interfaces with the sensory-motor nucleus HVC (used as proper name), where neurons generate high-level motor commands that shape the production of learned song. Adult zebra finches (Taeniopygia guttata) sing renditions of a stereotyped motif (a sequence of 3-10 syllables), whose temporal and/or motor structure is thought to be encoded in the activities of two major types of HVC neurons (Fig. 1a) [8][9][10][11][12][13] . We implanted 16/32 site Si-probes in male, adult zebra finches and recorded simultaneously their song and neural activity in HVC; then we used these data to train a long-short-term memory network (LSTM 5 ) to translate neural activity directly onto song. The goal of the network is to predict the spectral components of the song at a time bin t i , given the values of neural activity features over previous time bins (Fig. 1d-f). The neural activity is (t , t , )fed as a matrix comprising mean firing rates in each time bin, of each putative single/multi-unit automatically sorted from the recordings 14 (32/64 clusters); the spectral components of the song are represented by the power across 64 log-spaced frequency bands. For each session (day), we separate the 70-110 renditions of a motif the bird sang. We then train the LSTM network to find the neural-to-song spectrum mappings, and decode the corresponding spectral components from a test set of neural activity to finally recover waveforms of synthetically generated song motifs. We employed several methods to avoid overfitting. First, the order in which each pair of neural feature window/target was presented to the network was randomized, so that the predictions of the spectral components at each time point are independent; second, we used standard techniques such as L2 weight regularization, dropout and early stopping (see Methods). We also employed two different procedures to produce the training/validation and test sets. For motif-wise training/decoding, we split the data into peer-reviewed)

show abstract

“…Real biophysical parameters, of course, must be continuous functions of time, but when the researchers measured the bronchial pressure and syringeal muscle activity in singing birds, they found that both quantities exhibited rapid changes exactly where the model predicted. 4 A spectrograph of the reconstructed song is shown in figure 2c. (Sound files for both the recorded and reconstructed songs are available with the online version of this story.…”

mentioning

confidence: 99%