Semantically Congruent Bimodal Presentation with Divided-Modality Attention Accelerates Unisensory Working Memory Retrieval

Abstract

Although previous studies have shown that semantic multisensory integration can be differentially modulated by attention focus, it remains unclear whether attentionally mediated multisensory perceptual facilitation could impact further cognitive performance. Using a delayed matching-to-sample paradigm, the present study investigated the effect of semantically congruent bimodal presentation on subsequent unisensory working memory (WM) performance by manipulating attention focus. The results showed that unisensory WM retrieval was faster in the semantically congruent condition than in the incongruent multisensory encoding condition. However, such a result was only found in the divided-modality attention condition. This result indicates that a robust multisensory representation was constructed during semantically congruent multisensory encoding with divided-modality attention; this representation then accelerated unisensory WM performance, especially auditory WM retrieval. Additionally, an overall faster unisensory WM retrieval was observed under the modality-specific selective attention condition compared with the divided-modality condition, indicating that the division of attention to address two modalities demanded more central executive resources to encode and integrate crossmodal information and to maintain a constructed multisensory representation, leaving few resources for WM retrieval. Additionally, the present finding may support the amodal view that WM has an amodal central storage component that is used to maintain modal-based attention-optimized multisensory representations.

College English Multimodal Teaching Based on Digital Information Technology

“…In particular, English teaching in colleges and universities has been deeply integrated with multimedia teaching. Teachers basically use multimedia teaching mode in class [ 1 ]. However, although this teaching mode has made some progress with the traditional teaching mode, it still continues the drawbacks of the traditional teaching mode, especially the students' enthusiasm in the classroom and learning initiative.…”

Semantic Bimodal Presentation Differentially Slows Working Memory Retrieval

“…Working memory (WM) is typically considered a system with limited capacity that can process, store, and monitor information in a short period [1], which is the ability and/or resources to concurrently process and store task-relevant information or coordinate the processing of multiple competing inputs [2]. Some studies have reported that WM involves the temporal maintenance of an active representation of external perception information so that it is available for subsequent retrieval processing [3]. Recent evidence suggests that brains tend to aggregate and analyze information from different sources with connections to form a unified cognitive representation [4] and demonstrate a bimodal WM retrieval advantage [5].…”

“…The multisensory congruency relationship reflects the tendency to associate seemingly arbitrary features or stimulus dimensions that have a certain relationship under multisensory conditions [3,7]. Previous multisensory studies have indicated that multisensory integration efficiency was modulated by high-level semantic relationships between different sensory modalities [9].…”

“…Previous multisensory studies have indicated that multisensory integration efficiency was modulated by high-level semantic relationships between different sensory modalities [9]. Recently, many studies found that not only enhanced perceptual behavioral performance but also accelerated WM retrieval was observed when visual and auditory stimuli shared common rather than conflicting semantic information [3,8]. For example, Xie et al (2017) reported faster visual WM retrieval in a semantically congruent audiovisual WM encoding condition [10].…”

“…The effect of multisensory congruency on WM mainly focuses on semantic matching, such as picture and sound (e.g., a picture of a dog and the sound "woof" or a picture of a dog and an oral stimulus of "dog") [3,14] and written text and sound (e.g., the word "dog" with the sound "woof" or the word "dog" with the spoken word "dog") [15] and is less focused on matching other stimulus attributes. Even in studies of vocabulary, little consideration has been given to the impact of the audiovisual matching of lexical characteristics.…”