A Neural Network Approach to fMRI Binocular Visual Rivalry Task Analysis

Abstract

The purpose of this study was to investigate whether artificial neural networks (ANN) are able to decode participants’ conscious experience perception from brain activity alone, using complex and ecological stimuli. To reach the aim we conducted pattern recognition data analysis on fMRI data acquired during the execution of a binocular visual rivalry paradigm (BR). Twelve healthy participants were submitted to fMRI during the execution of a binocular non-rivalry (BNR) and a BR paradigm in which two classes of stimuli (faces and houses) were presented. During the binocular rivalry paradigm, behavioral responses related to the switching between consciously perceived stimuli were also collected. First, we used the BNR paradigm as a functional localizer to identify the brain areas involved the processing of the stimuli. Second, we trained the ANN on the BNR fMRI data restricted to these regions of interest. Third, we applied the trained ANN to the BR data as a ‘brain reading’ tool to discriminate the pattern of neural activity between the two stimuli. Fourth, we verified the consistency of the ANN outputs with the collected behavioral indicators of which stimulus was consciously perceived by the participants. Our main results showed that the trained ANN was able to generalize across the two different tasks (i.e. BNR and BR) and to identify with high accuracy the cognitive state of the participants (i.e. which stimulus was consciously perceived) during the BR condition. The behavioral response, employed as control parameter, was compared with the network output and a statistically significant percentage of correspondences (p-value <0.05) were obtained for all subjects. In conclusion the present study provides a method based on multivariate pattern analysis to investigate the neural basis of visual consciousness during the BR phenomenon when behavioral indicators lack or are inconsistent, like in disorders of consciousness or sedated patients.

fMRI-based decoding of reward effects in binocular rivalry

“…In the present study, we sought to extend this research by using another indirect measure of perception based on brain activation. To this aim, we applied fMRI-based multivoxel pattern analysis (MVPA) to decode perceptual fluctuations from neural activation in the visual cortex – a method that has previously proven useful for the tracking of perception during binocular rivalry ( Haynes and Rees 2005 ; Bertolino et al 2014 ) and other types of bistable perception ( Brouwer and van Ee 2007 ; Schmack et al 2013 ; Brascamp et al 2018 ). We hypothesized that such decoding of perceptual fluctuations would reveal an increase in dominance durations for the perceptual state paired with reward.…”

Reproducibility of importance extraction methods in neural network based fMRI classification

“…Neural network classifiers have been previously used to classify fMRI data either with hidden layers (Bertolino et al, 2014;Floren et al, 2015;Misaki et al, 2006) or without (Polyn et al, 2005;Saarimäki et al, 2016). The majority of MVPA studies use support vector classifiers (SVC) (Cox & Savoy, 2003;De Martino et al, 2008, Ethofer et al, 2009Habes et al, 2013;LaConte et al, 2005;Kamitani & Tong, 2005;Lahnakoski et al, 2014;Lie et al, 2013;Meier et al, 2012;Mourão-Miranda et al, 2005;Mourão-Miranda et al, 2007;Rasmussen et al, 2011; see also Sundermann et al, 2014, for an extended list) due to fast training and good performance in ill-posed problems such as in fMRI classification (Etzel et al, 2013).…”

Reproducibility of importance extraction methods in neural network based fMRI classification

“…Neural network classifiers have been previously used to classify fMRI data either with hidden layers (Bertolino et al, 2014;Floren et al, 2015;Misaki et al, 2006) or without (Polyn et al, 2005;Saarimäki et al, 2016). The majority of MVPA studies use support vector classifiers (SVC) (Cox & Savoy, 2003;De Martino et al, 2008, Ethofer et al, 2009Habes et al, 2013;LaConte et al, 2005;Kamitani & Tong, 2005;Lahnakoski et al, 2014;Lie et al, 2013;Meier et al, 2012;Mourão-Miranda et al, 2005;Mourão-Miranda et al, 2007;Rasmussen et al, 2011; see also Sundermann et al, 2014, for an extended list) due to fast training and good performance in ill-posed problems such as in fMRI classification (Etzel et al, 2013).…”