Isolating Neural Indices of Continuous Speech Processing at the Phonetic Level

Abstract

The human ability to understand speech across an enormous range of listening conditions is underpinned by a hierarchical auditory processing system whose successive stages process increasingly complex attributes of the acoustic input. In order to produce a categorical perception of words and phonemes, it has been suggested that, while earlier areas of the auditory system undoubtedly respond to acoustic differences in speech tokens, later areas must exhibit consistent neural responses to those tokens. Neural indices of such hierarchical processing in the context of continuous speech have been identified using low-frequency scalprecorded electroencephalography (EEG) data. The relationship between continuous speech and its associated neural responses has been shown to be best described when that speech is represented using both its low-level spectrotemporal information and also the categorical labelling of its phonetic features (Di Liberto et al., Curr Biol 25(19):2457-2465, 2015. While the phonetic features have been proven to carry extra-information not captured by the speech spectrotemporal representation, the causes of this EEG activity remain unclear. This study aims to demonstrate a framework for examining speech-specific processing and for disentangling highlevel neural activity related to intelligibility from low-level activity in response to spectrotemporal fluctuations of speech. Preliminary results suggest that neural measure of processing at the phonetic level can be isolated.

Cortical Measures of Phoneme-Level Speech Encoding Correlate with the Perceived Clarity of Natural Speech

“…This was calculated as the sum of the three band-limited envelopes that compose the S representation. In previous work, our framework has also included a phonemic representation of the speech (a multivariate time series of forced aligned phonemes, similar to F; Di Liberto and Lalor, 2016 ). However, because of the limited amount of speech data used in the present study, less frequent phonemes would not have a sufficient number of occurrences to produce a good model fit.…”

M/EEG analysis of naturalistic stories: a review from speech to language processing

“…In defining TRFs, it is also possible to include additional covariates, such as linguistic information. Di Liberto and colleagues (DiLiberto, O'Sullivan, & Lalor, 2015;Liberto & Lalor, 2016) took a modelbased approach to entrainment and found that the inclusion of phonemic labels in addition to the speech envelope improve the model's ability to predict EEG data in all frequency bands. Note phonemes are not present at any instant in the acoustic signal, so phonemic labels encode temporally diffuse aspects of the latent linguistic signal.…”

Predicting individual speech intelligibility from the cortical tracking of acoustic- and phonetic-level speech representations

“…However, while these results are promising, the correlation between the behavioral and objective measures of speech intelligibility only explained 50% of the variance, and the objective measure could only be derived in three-quarter of the subjects. Recently, Di Liberto et al (2015Liberto et al ( , 2016Liberto et al ( & 2017 showed that the cortical tracking of running speech is better characterized using a model integrating both low-level spectro-temporal speech information (e.g., the speech envelope) and discrete higher-level phonetic features.…”