Automated vocal analysis of naturalistic recordings from children with autism, language delay, and typical development

Abstract

For generations the study of vocal development and its role in language has been conducted laboriously, with human transcribers and analysts coding and taking measurements from small recorded samples. Our research illustrates a method to obtain measures of early speech development through automated analysis of massive quantities of day-long audio recordings collected naturalistically in children's homes. A primary goal is to provide insights into the development of infant control over infrastructural characteristics of speech through large-scale statistical analysis of strategically selected acoustic parameters. In pursuit of this goal we have discovered that the first automated approach we implemented is not only able to track children's development on acoustic parameters known to play key roles in speech, but also is able to differentiate vocalizations from typically developing children and children with autism or language delay. The method is totally automated, with no human intervention, allowing efficient sampling and analysis at unprecedented scales. The work shows the potential to fundamentally enhance research in vocal development and to add a fully objective measure to the battery used to detect speech-related disorders in early childhood. Thus, automated analysis should soon be able to contribute to screening and diagnosis procedures for early disorders, and more generally, the findings suggest fundamental methods for the study of language in natural environments.

Predicting the birth of a spoken word

“…Word frequencies can easily be calculated from transcripts (7,8), and overall quantity can even be estimated via automated methods (10). Sheer frequency may not be the best predictor of word learning, however.…”

Properties of vocalization- and gesture-combinations in the transition to first words

“…In general CVIs correspond to syllables with very strong differentiations of acoustic energy level between nuclei (or vowels) and margins (or consonants) (Oller et al 2010). Given that in our sample vegetative sounds and cries were excluded, all the CVIs obtained were speech-related vocal islands (SVI), following Oller et al's (2010) classification. Analysis of SVIs focused on acoustic effects of rhythmic "movements" of jaw, tongue, and lips (i.e., articulation), which underlie syllabic organization, and on acoustic effects of vocal quality or "voice" (Oller et al, 2010).…”

“…A child vocal island was identified when the acoustic energy level rose to 90% above baseline for at least 50 ms and ended when it fell to less than 10% above baseline for at least 50 ms, but not more than 300 ms.. In general CVIs correspond to syllables with very strong differentiations of acoustic energy level between nuclei (or vowels) and margins (or consonants) (Oller et al 2010). Given that in our sample vegetative sounds and cries were excluded, all the CVIs obtained were speech-related vocal islands (SVI), following Oller et al's (2010) classification.…”

“…Eleven Spanish children aged 0;9 to 1;3 were observed longitudinally in a semi-structured play situation with an adult. Vocalizations were analyzed using several acoustic parameters based on those described by Oller et al (2010). Results indicate that declarative vocalizations have less protosyllables than imperative ones, but only when they are produced with a gesture.…”

“…Unlike the case of previous studies, we performed a thorough analysis of the acoustic parameters of vocalizations, so as to investigate whether vocal-gestural combination has an impact on vocal characteristics. Our analysis strategy was based on the parameters and categories defined by Oller et al (2010), which have been sensitive to developmental changes and relevant in differentiating children with and without disorders such as autism or language delay. Critically, these parameters allow us to Running head: VOCALIZATIONS AND GESTURAL COMBINATION 9 detect "speech-related vocal islands" (SVIs), which can be considered as precursors of mature syllables.…”