Onscreen presence of instructors in video lectures affects learners’ neural synchrony and visual attention during multimedia learning

Abstract

COVID-19 forced students to rely on online learning using multimedia tools, and multimedia learning continues to impact education beyond the pandemic. In this study, we combined behavioral, eye-tracking, and neuroimaging paradigms to identify multimedia learning processes and outcomes. College students viewed four video lectures including slides with either an onscreen human instructor, an animated instructor, or no onscreen instructor. Brain activity was recorded via fMRI, visual attention was recorded via eye-tracking, and learning outcome was assessed via post-tests. Onscreen presence of instructor, compared with no instructor presence, resulted in superior post-test performance, less visual attention on the slide, more synchronized eye movements during learning, and higher neural synchronization in cortical networks associated with socio-emotional processing and working memory. Individual variation in cognitive and socio-emotional abilities and intersubject neural synchronization revealed different levels of cognitive and socio-emotional processing in different learning conditions. The instructor-present condition evoked increased synchronization, likely reflecting extra processing demands in attentional control, working memory engagement, and socio-emotional processing. Although human instructors and animated instructors led to comparable learning outcomes, the effects were due to the dynamic interplay of information processing vs. attentional distraction. These findings reflect a benefit–cost trade-off where multimedia learning outcome is enhanced only when the cognitive benefits motivated by the social presence of onscreen instructor outweigh the cognitive costs brought about by concurrent attentional distraction unrelated to learning.

Effect of Explanation Videos on Learning: The Role of Attention and Academic Performance

“…Our result here appears to contradict previous studies that have shown a positive correlation between eye ISC during videos and performance on subsequent quizzes ( [42], [55]). We did measure the predicted effect of ISC for the matching videos, namely, the attention to the matching video improved performance.…”

“…A number of studies using EEG showed that modulating the attentional state of the viewer will modulate ISC [40] and therefore ISC is predictive of memory [41], including in the context of short instructional videos [41]. Madsen et al extended these findings by showing that synchronized eye movements during video viewing are modulated by attention predictive of test scores [15], a result recently validated by an independent research group [42]. These studies collectively suggest that the level of attention to short educational videos is quantifiable and can be measured online utilizing webcam eye trackers.…”