Machine Learning Approaches to Analyze Speech-Evoked Neurophysiological Responses

Abstract

Purpose Speech-evoked neurophysiological responses are often collected to answer clinically and theoretically driven questions concerning speech and language processing. Here, we highlight the practical application of machine learning (ML)–based approaches to analyzing speech-evoked neurophysiological responses. Method Two categories of ML-based approaches are introduced: decoding models, which generate a speech stimulus output using the features from the neurophysiological responses, and encoding models, which use speech stimulus features to predict neurophysiological responses. In this review, we focus on (a) a decoding model classification approach, wherein speech-evoked neurophysiological responses are classified as belonging to 1 of a finite set of possible speech events (e.g., phonological categories), and (b) an encoding model temporal response function approach, which quantifies the transformation of a speech stimulus feature to continuous neural activity. Results We illustrate the utility of the classification approach to analyze early electroencephalographic (EEG) responses to Mandarin lexical tone categories from a traditional experimental design, and to classify EEG responses to English phonemes evoked by natural continuous speech (i.e., an audiobook) into phonological categories (plosive, fricative, nasal, and vowel). We also demonstrate the utility of temporal response function to predict EEG responses to natural continuous speech from acoustic features. Neural metrics from the 3 examples all exhibit statistically significant effects at the individual level. Conclusion We propose that ML-based approaches can complement traditional analysis approaches to analyze neurophysiological responses to speech signals and provide a deeper understanding of natural speech and language processing using ecologically valid paradigms in both typical and clinical populations.

Evolving perspectives on the sources of the frequency-following response

“…Combined with free-field recordings 91 , portable FFR systems 92 , and/or wearable technologies 93 , these methods open opportunities to examine FFR in real-world settings. On the analytical front, machine learning algorithms have recently been developed allowing single-trial FFR classification 94,95 which could have many applications, including for instance as neurofeedback in training paradigms.…”

Infants’ neural speech discrimination predicts individual differences in grammar ability at 6 years of age and their risk of developing speech-language disorders

“…In such cases, the mismatch response time series should bear no predictive value to SPELT score and the R 2 should reflect a model performing at chance level. We repeated this process 1000 times and generated an empirical null distribution of R 2 and we compared our originally obtained R 2 coefficient against this distribution ( Xie et al, 2019 ).…”

Rapid Enhancement of Subcortical Neural Responses to Sine-Wave Speech

“…Previous experiments have used machine learning algorithms to assess whether the information contained in FFRs is sufficient to decode the stimulus classes that evoked them (Sadeghian et al, 2015;Holdgraf et al, 2017;Llanos et al, 2017;Yi et al, 2017;Xie et al, 2018Xie et al, , 2019. Under this approach, FFR classification performance (i.e., the accuracy with which FFRs are correctly classified by the machine learning algorithm) serves as an objective measure of stimulus discrimination.…”

“…A MATLAB-constructed linear support vector machine (SVM; Cristianini and Shawe-Taylor, 2000) was used to classify pre-and post-training FFR SWS for test and control groups following the general procedures described by Xie et al (2019). We first epoched all subtracted FFR SWS waveforms from 0 to 380 ms and used these 1,900 amplitude-by-time points as linear SVM input features.…”

“…The model was cross-validated using a three-fold approach that was repeated 2,500 times (see Xie et al, 2019; Figure 1). For each iteration of the linear SVM classifier, participants were randomly and equally divided into 3 groups (or folds).…”