Observation and assessment of acoustic contamination of electrophysiological brain signals during speech production and sound perception

Abstract

Objective. A current challenge of neurotechnologies is to develop speech brain-computer interfaces aiming at restoring communication in people unable to speak. To achieve a proof of concept of such system, neural activity of patients implanted for clinical reasons can be recorded while they speak. Using such simultaneously recorded audio and neural data, decoders can be built to predict speech features using features extracted from brain signals. A typical neural feature is the spectral power of field potentials in the high-gamma frequency band, which happens to overlap the frequency range of speech acoustic signals, especially the fundamental frequency of the voice. Here, we analyzed human electrocorticographic and intracortical recordings during speech production and perception as well as a rat microelectrocorticographic recording during sound perception. We observed that several datasets, recorded with different recording setups, contained spectrotemporal features highly correlated with those of the sound produced by or delivered to the participants, especially within the high-gamma band and above, strongly suggesting a contamination of electrophysiological recordings by the sound signal. This study investigated the presence of acoustic contamination and its possible source. Approach. We developed analysis methods and a statistical criterion to objectively assess the presence or absence of contamination-specific correlations, which we used to screen several datasets from five centers worldwide. Main results. Not all but several datasets, recorded in a variety of conditions, showed significant evidence of acoustic contamination. Three out of five centers were concerned by the phenomenon. In a recording showing high contamination, the use of high-gamma band features dramatically facilitated the performance of linear decoding of acoustic speech features, while such improvement was very limited for another recording showing no significant contamination. Further analysis and in vitro replication suggest that the contamination is caused by the mechanical action of the sound waves onto the cables and connectors along the recording chain, transforming sound vibrations into an undesired electrical noise affecting the biopotential measurements. Significance. Although this study does not per se question the presence of speech-relevant physiological information in the high-gamma range and above (multiunit activity), it alerts on the fact that acoustic contamination of neural signals should be proofed and eliminated before investigating the cortical dynamics of these processes. To this end, we make available a toolbox implementing the proposed statistical approach to quickly assess the extent of contamination in an electrophysiological recording (https://doi.org/10.5281/zenodo.3929296).

Differentiation of speech-induced artifacts from physiological high gamma activity in intracranial recordings

“…Fourth, significant inter-trial phase consistency between the produced audio and the neural data suggests similarities not only across the frequency domain, but also the temporal domain (Figure 5). Fifth, most of the recordings that we classified as contaminated in our analysis were also classified as having acoustic contamination by the recently proposed method in Roussel et al (2020) (Figure 6 and Table 2). Sixth, there was no significant cortical localization of the ITPC index (Figure 7).…”

“…It occurs almost exclusively during cued speech production epochs and it has spectral and temporal characteristics strikingly similar to the produced speech audio. Indeed, a recent work shows the presence of acoustic contamination on ECoG recordings (Roussel et al, 2020). In the environment of DBS surgery, speech-induced vibrations conducted by the skull and stereotactic frame can impinge on the electrodes and/or signal acquisition chain affecting the recorded signal.…”

“…The spectrogram correlation index (Figure 4) and the coherence index (Figure 5) are highly correlated with each other, as can be observed in Figure 6A (Spearman's ⍴ = 0.7, p < 1e-6). In applying the method proposed by Roussel et al (2020), the power of the neural data was correlated with the audio across different frequency bins to create a correlation matrix (Figure 6B). High correlation coefficients on the diagonal of this matrix indicate acoustic contamination (see details in the Methods Section).…”

Imagined speech can be decoded from low- and cross-frequency features in perceptual space

“…The absence of behavioral output during imagined speech might even be an advantage, as it definitely prevents the contamination of neural signal recordings by the participant's voice, a serious problem that was recently discovered. Because the fundamental frequency of the human voice overlaps with the neural BHA, an acousto-electric effect might have artificially inflated the performance in previous overt speech decoding studies (Roussel et al, 2020). To enable a fair comparison of overt and imagined speech in our study, we took care of checking that the three current datasets were free of acoustic contamination.…”

“…Time series were visually inspected, and contacts or trials containing epileptic activity and excessive noise were removed. Trials with overt speech were checked for acoustic contamination by correlating the recorded audio signal and the neural data (Roussel et al, 2020). All times series were then corrected for DC shifts by using a high-pass filter with a cutoff frequency of 0.5 Hz (zero-phase Butterworth filter of order 6, zeropole-gain design).…”