Dynamic cognitive processes of text-picture integration revealed by event-related potentials

Li, Songqing; Chen, Shi; Zhang, Hongpo; Zhao, Qiancheng; Zhou, Zhiyong; Huang, Furong; Sui, Danni; Wang, Fuxing; Hong, Jianzhong

doi:10.1016/j.brainres.2019.146513

Cited by 6 publications

(3 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shan et al found that the relevant brain areas for Chinese semantic processing are the left lower frontal gyrus, the left posterior inferior temporal gyrus, the joint area of the lower parietal lobe, and the superior temporal gyrus [35]. In the study by Song et al, they found that a late positive component was induced in the central, parietal, and temporal regions, which they believed might be related to the semantic activation and integration of text and pictures, respectively [34].…”

Section: E Difference In Brain Response Between Neutral and Phase Random Stimulusmentioning

confidence: 99%

“…e results of the difference in source response between neutral and phase random stimulus showed that, in the time period between 0.5 s and 1 s, the degree of activation of the occipital, prefrontal, and temporal lobes on the left and right sides of the neutral emotion was greater than that of phase random picture evoked emotion. Song et al studied the difference between text and picture integration process; they found that in the early period of picture processing a larger negative N1 was evoked in the occipital region, and a negative value of N300 was evoked in the prefrontal area, which may reflect the identification process of visual stimuli and the image representation of the picture [34]. After 1.5 s of stimulation, the EEG source response intensities induced by phase random stimulation in the anterior parietal lobe of the left and right hemispheres gradually enhanced and were greater than those of neutral emotion.…”

Section: E Difference In Brain Response Between Neutral and Phase Random Stimulusmentioning

confidence: 99%

See 1 more Smart Citation

Processing of Affective Pictures: A Study Based on Functional Connectivity Network in the Cerebral Cortex

Yang

Zhuang

et al. 2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Emotion plays an important role in people’s life. However, the existing researches do not give a unified conclusion on the brain function network under different emotional states. In this study, pictures from the international affective picture system (IAPS) of different valences were presented to subjects with a fixed frequency blinking frequency to induce stable state visual evoked potential (SSVEP). With the source location method, the cerebral cortex source signal was reconstructed based on EEG signals, and then the difference in SSVEP amplitudes in key brain areas under different emotional states and the difference in brain function network connections among different brain areas were analysed in cortical space. The results of the study show that positive and negative emotions evoked greater activation intensities in the prefrontal, temporal, and parietal lobes compared with those of neutral emotion. The network connections with a significant difference between emotional states mainly appear in the alpha and gamma bands, and the network connections with significant differences between positive emotion and negative emotion mainly exist in the right middle temporal gyrus and the superior frontal gyrus on both sides. In addition, the long-range connections play an important role in the process of emotional processing, especially the connections among frontal gyrus, middle temporal gyrus, and middle occipital gyrus. The results of this study provide a reliable scientific basis for revealing and elucidating the neural mechanism of emotion processing and the selection of brain regions and frequency bands in emotion recognition based on EEG signals.

show abstract

Section: E Difference In Brain Response Between Neutral and Phase Random Stimulusmentioning

confidence: 99%

Section: E Difference In Brain Response Between Neutral and Phase Random Stimulusmentioning

confidence: 99%

Processing of Affective Pictures: A Study Based on Functional Connectivity Network in the Cerebral Cortex

Yang

Zhuang

et al. 2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Due to the high temporal resolution of EEG, Event-Related Potentials (ERP) studies can also tell us a lot about the temporal dynamics of the brain activity, particularly by repeatedly presenting conditions time-locked to the stimulus onset. The timing of text-picture integration was explored in ERP and ERD/ERS studies 9 . By plotting the time-frequency of the and channels, Scharinger concluded that EEG is a valid and practical tool to measure mental processing demand 10 .…”

Section: Introductionmentioning

confidence: 99%

Effects and prediction of cognitive load on encoding model of brain response to auditory and linguistic stimuli in educational multimedia

Asaadi,

Amiri,

Bosaghzadeh

et al. 2024

Sci Rep

View full text Add to dashboard Cite

Multimedia is extensively used for educational purposes. However, certain types of multimedia lack proper design, which could impose a cognitive load on the user. Therefore, it is essential to predict cognitive load and understand how it impairs brain functioning. Participants watched a version of educational multimedia that applied Mayer’s principles, followed by a version that did not. Meanwhile, their electroencephalography (EEG) was recorded. Subsequently, they participated in a post-test and completed a self-reported cognitive load questionnaire. The audio envelope and word frequency were extracted from the multimedia, and the temporal response functions (TRFs) were obtained using a linear encoding model. We observed that the behavioral data are different between the two groups and the TRFs of the two multimedia versions were different. We saw changes in the amplitude and latencies of both early and late components. In addition, correlations were found between behavioral data and the amplitude and latencies of TRF components. Cognitive load decreased participants’ attention to the multimedia, and semantic processing of words also occurred with a delay and smaller amplitude. Hence, encoding models provide insights into the temporal and spatial mapping of the cognitive load activity, which could help us detect and reduce cognitive load in potential environments such as educational multimedia or simulators for different purposes.

show abstract

Using eye-tracking and EEG to study the mental processing demands during learning of text-picture combinations

Scharinger

Schüler

Gerjets

2020

International Journal of Psychophysiology

View full text Add to dashboard Cite

Dynamic cognitive processes of text-picture integration revealed by event-related potentials

Cited by 6 publications

References 45 publications

Processing of Affective Pictures: A Study Based on Functional Connectivity Network in the Cerebral Cortex

Processing of Affective Pictures: A Study Based on Functional Connectivity Network in the Cerebral Cortex

Effects and prediction of cognitive load on encoding model of brain response to auditory and linguistic stimuli in educational multimedia

Using eye-tracking and EEG to study the mental processing demands during learning of text-picture combinations

Contact Info

Product

Resources

About