Decoding power-spectral profiles from FMRI brain activities during naturalistic auditory experience

Abstract

Recent studies have demonstrated a close relationship between computational acoustic features and neural brain activities, and have largely advanced our understanding of auditory information processing in the human brain. Along this line, we proposed a multidisciplinary study to examine whether power spectral density (PSD) profiles can be decoded from brain activities during naturalistic auditory experience. The study was performed on a high resolution functional magnetic resonance imaging (fMRI) dataset acquired when participants freely listened to the audio-description of the movie "Forrest Gump". Representative PSD profiles existing in the audio-movie were identified by clustering the audio samples according to their PSD descriptors. Support vector machine (SVM) classifiers were trained to differentiate the representative PSD profiles using corresponding fMRI brain activities. Based on PSD profile decoding, we explored how the neural decodability correlated to power intensity and frequency deviants. Our experimental results demonstrated that PSD profiles can be reliably decoded from brain activities. We also suggested a sigmoidal relationship between the neural decodability and power intensity deviants of PSD profiles. Our study in addition substantiates the feasibility and advantage of naturalistic paradigm for studying neural encoding of complex auditory information.

A studyforrest extension, retinotopic mapping and localization of higher visual areas

“…The dataset has, so far, been used to study the role of the insula in dynamic emotional experiences 2 , modeling of shared blood oxygenation level dependent (BOLD) response patterns across brains 3 , and to decode input audio power-spectrum profiles from fMRI 4 . The dataset has subsequently been extended twice, first with additional fMRI data from stimulation with music from various genres 5 and secondly with a description of the movie stimulus structure with respect to portrayed emotions 6 .…”

A Set of Functional Brain Networks for the Comprehensive Evaluation of Human Characteristics

“…Using behavioral data about psychometric parameters as the supporting information, we attempted to test whether functional networks of the brain could be identified by rs-fMRI signals and psychological parameters (Figure 1 ). Recently some studies have shown that it is possible to decode brain states from fMRI responses using machine learning algorithms, such as a support vector machine (Guo et al, 2015 ; Altmann et al, 2016 ; Hu et al, 2017 ; Zafar et al, 2017 ). To verify that the functional networks identified represent the corresponding psychological parameters, and also to evaluate human characteristics from the rs-fMRI signals, we used a support vector machine (SVM).…”

Training neural networks to recognize speech increased their correspondence to the human auditory pathway but did not yield a shared hierarchy of acoustic features

“…It's been shown that single trial (i.e. without repetition) measurements during movie watching contain sufficient information to train successful decoding models (Hu et al, 2017) and that functional alignment across subjects based on such single trial measurements can improve decoding performance relative to single-subject decoding (Haxby et al, 2011, Bazeille et al, 2020.…”

“…The copyright holder for this preprint this version posted January 27, 2021. ; https://doi.org/10.1101/2021.01.26.428323 doi: bioRxiv preprint movie watching contain sufficient information to train successful decoding models (Hu et al, 2017) and that functional alignment across subjects based on such single trial measurements can improve decoding performance relative to single-subject decoding (Haxby et al, 2011, Bazeille et al, 2020. Experimental designs of this type sacrifice reliable responses to individual conditions in favor of maximizing the diversity of stimuli presented (which aids generalization) and the number of brain volumes collected.…”