Multisensory integration of dynamic emotional faces and voices: method for simultaneous EEG-fMRI measurements

Abstract

Combined EEG-fMRI analysis correlates time courses from single electrodes or independent EEG components with the hemodynamic response. Implementing information from only one electrode, however, may miss relevant information from complex electrophysiological networks. Component based analysis, in turn, depends on a priori knowledge of the signal topography. Complex designs such as studies on multisensory integration of emotions investigate subtle differences in distributed networks based on only a few trials per condition. Thus, they require a sensitive and comprehensive approach which does not rely on a-priori knowledge about the underlying neural processes. In this pilot study, feasibility and sensitivity of source localization-driven analysis for EEG-fMRI was tested using a multisensory integration paradigm. Dynamic audiovisual stimuli consisting of emotional talking faces and pseudowords with emotional prosody were rated in a delayed response task. The trials comprised affectively congruent and incongruent displays. In addition to event-locked EEG and fMRI analyses, induced oscillatory EEG responses at estimated cortical sources and in specific temporo-spectral windows were correlated with the corresponding BOLD responses. EEG analysis showed high data quality with less than 10% trial rejection. In an early time window, alpha oscillations were suppressed in bilateral occipital cortices and fMRI analysis confirmed high data quality with reliable activation in auditory, visual and frontal areas to the presentation of multisensory stimuli. In line with previous studies, we obtained reliable correlation patterns for event locked occipital alpha suppression and BOLD signal time course. Our results suggest a valid methodological approach to investigate complex stimuli using the present source localization driven method for EEG-fMRI. This novel procedure may help to investigate combined EEG-fMRI data from novel complex paradigms with high spatial and temporal resolution.

The Shape of Water Stream Induces Differences in P300 and Alpha Oscillation

“…Bauer et al (2006) reported that parieto-occipital alpha/beta suppression was modulated by spatial attention. This component over the parieto-occipital distribution has been observed in response to audiovisual stimulation, suggesting that this component is generated by cognitive processes independent of sensory modality (Schelenz et al, 2013). Some studies demonstrated that this component was more strongly suppressed when participants perceived a negative emotional stimulus compared to when perceiving a neutral stimulus (Jessen and Kotz, 2011;Swingle, 2013).…”

Recent advances in laboratory assessment of emotion regulation

“…EEG may therefore yield useful information about brain network activity from larger samples and in settings when fMRI is either impractical or prohibitively costly. However, acquiring data in both modalities (i.e., “combined EEG-fMRI” [33]) holds promise to better capture implicit and explicit processes in emotion regulation. The ability to integrate multiple streams of information is integral to processing [34] as well as regulating emotions, as it is rare that evocative situations and events present themselves in only one sensory modality.…”

“…Optimal regulation involves selecting a strategy that is not only effective, but appropriate to the given environmental context. EEG-fMRI may facilitate the use of complex stimuli in emotion regulation research, given its potential to parse the temporal and spatial pathways involved in crossmodal sensory integration in the brain [33]. …”

“…For example, one recent study found that startle reflex, LPP amplitude and threat stimuli (images of fearful facial expressions) were modulated by social anxiety in the presence of chemosensory stimuli – in this case, smelling the sweat of anxious individuals [55]. Methods such as combined EEG-fMRI that can capture both the time course and spatial pathways in how multiple sources of sensory information are organized [33] in service of regulation have potential to aid ecological validity. However, in order to maximize the utility of such research, there is a need to standardize procedures for collection, analysis and publication of data so that findings from different work groups may be compared or combined [3,39].…”

Neural processing of emotion in multimodal settings

“…The new technique of simultaneous EEG and fMRI recordings may adequately address this issue. Schelenz et al (2013) present a novel source-localization driven analysis for EEG-informed fMRI. Applied to multisensory emotion paradigms, this method has the potential to map the exact cortical pathways of audiovisual signal integration.…”