Identifying Object Categories from Event-Related EEG: Toward Decoding of Conceptual Representations

Abstract

Multivariate pattern analysis is a technique that allows the decoding of conceptual information such as the semantic category of a perceived object from neuroimaging data. Impressive single-trial classification results have been reported in studies that used fMRI. Here, we investigate the possibility to identify conceptual representations from event-related EEG based on the presentation of an object in different modalities: its spoken name, its visual representation and its written name. We used Bayesian logistic regression with a multivariate Laplace prior for classification. Marked differences in classification performance were observed for the tested modalities. Highest accuracies (89% correctly classified trials) were attained when classifying object drawings. In auditory and orthographical modalities, results were lower though still significant for some subjects. The employed classification method allowed for a precise temporal localization of the features that contributed to the performance of the classifier for three modalities. These findings could help to further understand the mechanisms underlying conceptual representations. The study also provides a first step towards the use of concept decoding in the context of real-time brain-computer interface applications.

Improving N1 classification by grouping EEG trials with phases of pre-stimulus EEG oscillations

“…The phase bifurcation effect in this time-frequency range had anterior-parietal topography with the top value at channel POz (Simanova et al 2010) which was reported connecting with N170. The white marker in Fig.…”

Auditory perceptual decision-making based on semantic categorization of environmental sounds

“…Using electrophysiologic signals, decoding sound categories at the single-trial AEP level has turned out to be more challenging than in the visual domain. Simanova et al (Simanova et al, 2010) attempted at decoding semantic categories of auditory stimuli at the single-trial level in an experiment in which subjects were asked to perform a task irrelevant to the categorization. A multivariate pattern analysis based on Bayesian logistic regression performed poorly in comparison to a similar analysis applied to visual stimuli; and in about half of the subjects they obtained chance-level classification accuracy.…”

“…Here, a popular paradigm involves discriminating the direction of motion of a field of moving dots (Bennur and Gold, 2011;Britten et al, 1992;Kim and Shadlen, 1999;Newsome et al, 1989;Shadlen and Newsome, 1996) during which activity in area MT could predict motion discrimination, whereas activity in the lateral intraparietal area (LIP), the frontal eye-field (FEF) and the dorsolateral prefrontal cortex (DLPFC) was directly related to forming the decision. More recently, perceptual decision-making in the visual domain has been investigated with complex stimuli and tasks involving perceptual and semantic categorization (Murphy et al 2011;Ratcliff et al, 2009;Simanova et al, 2010).…”

Predicting the Semantic Category of Internally Generated Words from Neuromagnetic Recordings

“…Developmental studies show that perceptual categorization develops during the first months of life (Mandler, 2004), and multiple neuroimaging studies have reported differential activation for stimuli from different semantic domains (Mahon & Caramazza, 2009;Gerlach, 2007;Martin & Chao, 2001;Chao & Martin, 2000;Chao, Haxby, & Martin, 1999;Caramazza & Shelton, 1998;Martin, Wiggs, Ungerleider, & Haxby, 1996;Perani et al, 1995). More recent research has demonstrated that the semantic category or entity of an object can be successfully predicted from neural activity patterns when the object is presented visually (Murphy et al, 2011;Reddy & Kanwisher, 2007;Haynes & Rees, 2006;Kamitani & Tong, 2005;Cox & Savoy, 2003;Haxby et al, 2001) or orthographically (Simanova, Hagoort, Oostenveld, & van Gerven, 2014;Chan, Halgren, Marinkovic, & Cash, 2011;Murphy et al, 2011;Shinkareva, Malave, Mason, Mitchell, & Just, 2011;Simanova, van Gerven, Oostenveld, & Hagoort, 2010). It can be argued, however, that the use of visual or orthographical stimuli in these tasks introduce confounding visual or phonological effects related to semantic retrieval (Hwang, Palmer, Basho, Zadra, & Müller, 2009).…”

“…It can be argued, however, that the use of visual or orthographical stimuli in these tasks introduce confounding visual or phonological effects related to semantic retrieval (Hwang, Palmer, Basho, Zadra, & Müller, 2009). An additional concern, more specific to visual stimulus presentation, is that certain perceptual attributes that are different between categories can bias the decoding outcome ( Vindiola & Wolmetz, 2011;Simanova et al, 2010). It is possible to minimize these confounds by using an internally guided word generation task, without presenting any pictures or words to participants.…”