N1 enhancement in synesthesia during visual and audio–visual perception in semantic cross-modal conflict situations: an ERP study

Abstract

Synesthesia entails a special kind of sensory perception, where stimulation in one sensory modality leads to an internally generated perceptual experience of another, not stimulated sensory modality. This phenomenon can be viewed as an abnormal multisensory integration process as here the synesthetic percept is aberrantly fused with the stimulated modality. Indeed, recent synesthesia research has focused on multimodal processing even outside of the specific synesthesia-inducing context and has revealed changed multimodal integration, thus suggesting perceptual alterations at a global level. Here, we focused on audio–visual processing in synesthesia using a semantic classification task in combination with visually or auditory–visually presented animated and in animated objects in an audio–visual congruent and incongruent manner. Fourteen subjects with auditory-visual and/or grapheme-color synesthesia and 14 control subjects participated in the experiment. During presentation of the stimuli, event-related potentials were recorded from 32 electrodes. The analysis of reaction times and error rates revealed no group differences with best performance for audio-visually congruent stimulation indicating the well-known multimodal facilitation effect. We found enhanced amplitude of the N1 component over occipital electrode sites for synesthetes compared to controls. The differences occurred irrespective of the experimental condition and therefore suggest a global influence on early sensory processing in synesthetes.

Developing synaesthesia: a primer

“…They found no support, neither for the serotinergic hyperactivity hypothesis nor for a related, reduced GABA levels hypothesis. Similarly, null findings were reported by Sinke et al ( 2014 ) in an investigation on the relationship between multisensory integration and event related potentials. Nevertheless, consistent with previous research, they found evidence for alterations in early visual processing in synaesthesia.…”

Reward enhances cross‐modal conflict control in object categorization: Electrophysiological evidence

“…A large number of previous studies have shown that N400 is implicated in semantic processing of objects in linguistic (Kutas & Federmeier, ) and pictorial (Ganis & Kutas, ) contexts. Previous studies also showed that N400 is related to cross‐modal semantic matching (Molholm et al, ; Schneider, Debener, Oostenveld, & Engel, ; Sinke et al, ). For example, Molholm et al () investigated audiovisual object recognition processes by using images and vocalizations of animals.…”

The brain of synesthetes

“…EEG and magnetencephalographic (MEG) methods have been used to delineate the time course of neurophysiological activation during synesthetic perceptions because they allow the measurement of cortical activations on a millisecond basis. Two types of studies have thus far used these techniques in synesthesia research: the first used measures of evoked potentials time-locked to the inducing stimuli (Schiltz et al 1999;Goller et al 2009;Brang et al 2008Brang et al , 2010Beeli et al 2008;J äncke et al 2012;Volberg et al 2013;Sinke et al 2014), which have been either visual (e.g., graphemes for grapheme-color synesthetes) or auditory stimuli (tones or words for auditory-color graphemes), and the second type used priming techniques and measured higher-order cognitive processes in the context of synesthetic experience (Gebuis et al 2009;Niccolai et al 2012b).…”