Acoustic contamination of electrophysiological brain signals during speech production and sound perception

Abstract

A current challenge of neurotechnologies is the development of speech brain-computer interfaces to restore communication in people unable to speak. To achieve a proof of concept of such system, neural activity of patients implanted for clinical reasons can be recorded while they speak. Using such simultaneously recorded audio and neural data, decoders can be built to predict speech features using features extracted from brain signals. A typical neural feature is the spectral power of field potentials in the high-gamma frequency band (between 70 and 200 Hz), a range that happens to overlap the fundamental frequency of speech. Here, we analyzed human electrocorticographic (ECoG) and intracortical recordings during speech production and perception as well as rat microelectrocorticographic (µ-ECoG) recordings during sound perception. We observed that electrophysiological signals, recorded with different recording setups, often contain spectrotemporal features highly correlated with those of the sound, especially within the high-gamma band. The characteristics of these correlated spectrotemporal features support a contamination of electrophysiological recordings by sound. In a recording showing high contamination, using neural features within the high-gamma frequency band dramatically increased the performance of linear decoding of acoustic speech features, while such improvement was very limited for another recording showing weak contamination. Further analysis and in vitro replication suggest that the contamination is caused by a mechanical action of the sound waves onto the cables and connectors along the recording chain, transforming sound vibrations into an undesired electrical noise that contaminates the biopotential measurements. This study does not question the existence of relevant physiological neural information underlying speech production or sound perception in the high-gamma frequency band, but alerts on the fact that care should be taken to evaluate and eliminate any possible acoustic contamination of neural signals in order to investigate the cortical dynamics of these processes.

Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus

“…To compare spectral content between recorded audio and electrodes ( Fig. 5A,B) , we convolved the voltage time series of each electrode and also the audio channel with a 200 ms Hamming window and then computed the power spectral density (PSD) in non-overlapping bins using a short-time Fourier transform (as in Roussel et al 2019). We isolated 'voicing epochs' in which to compare audio and neural power time series by sub-selecting time points with summed audio power (across all frequencies) in the top~10% of values across all audio data.…”

“…This in turn can exaggerate neural differences between phonemes by artificially shifting what are actually condition-invariant neural signal components. The second confound, which was recently raised by Roussel and colleagues (Roussel et al, 2019) , is that mechanical vibrations due to speaking might cause microphonic artifacts in the neural recordings. Our analyses suggest that while these confounds most likely do inflate speech decoding performance, their effects are not large.…”

High‐density intracranial recordings reveal a distinct site in anterior dorsal precentral cortex that tracks perceived speech

“…Heeding this report, we examined our results in this light. We did not find evidence for presence of a mechanical-electrical artifact, given the observations that (a) the ECoG-to-audio correlation we report showed a lot of variability across different parts of the film, which would not be expected if acoustic waves (present throughout the movie) were driving ECoG signals; (b) ECoG-to-audio correlations were only significant at a temporal lag of up to 300 ms (Figures 2a and S5); (c) the effects we report are present at a lower frequency range than the 115 Hz and up that Roussel et al (2019) report (recalculated and shown in Figure S5).…”

“…A recent report (Roussel et al, 2019) raised a possibility that audio signals may affect the integrity of ECoG data, because of a specific wiring setup and injection of mechanically‐induced electrical noise. Heeding this report, we examined our results in this light.…”

“…4.4 | Addressing the possibility of acoustic contamination in the ECoG signal A recent report (Roussel et al, 2019) raised a possibility that audio signals may affect the integrity of ECoG data, because of a specific wiring setup and injection of mechanically-induced electrical noise.…”