Multiple-Time-Scale Analysis of Attention as Revealed by EEG, NIRS, and Pupil Diameter Signals During a Free Recall Task: A Multimodal Measurement Approach

Abstract

Attention plays a fundamental role in acquiring and understanding information. Therefore, it is useful to evaluate attention objectively in such fields as education and mental health. Aimed at extracting objective indicators of attention from physiological signals, this study examined the characteristics of electroencephalography (EEG), near-infrared spectroscopy (NIRS), and pupil diameter signals during a free recall task. The objective was to clarify the temporal characteristics of these signals in relation to attention. We used a free recall task as a cognitive task with an attentional load. The participants attempted to memorize and then recall 13 serially presented words. Our hypothesis was that the significant physiological responses should differ depending on the time scale of the attention evaluation. The physiological responses were compared on the basis of differences between success and failure to recall a word on a short time scale, in terms of the attentional state among five serial position groups on a middle time scale, and on the basis of differences between trials with many and few words recalled on a long time scale. We found that the response of each physiological signal depended on the attention in the different time-scale comparisons. (1) The P300 amplitudes of the EEG signals for the words that were recalled were significantly higher than those for the words that were not recalled. (2) Pupillary dilation differed significantly depending on the serial position group. (3) Functional connectivity in the right hemisphere revealed by NIRS was significantly stronger in trials with many words recalled than in those with few words recalled. Different temporal characteristics of physiological signals with respect to attention were suggested by multimodal measurement and multiple-time-scale analysis. Consideration of these characteristics should help in the development of applications requiring objective attention evaluation.

Detecting Fear of Heights Response to a Virtual Reality Environment Using Functional Near-Infrared Spectroscopy

“…Nevertheless, it was also reported in the literature that Δ[HbO] values over the PFC can increase when the participants were experiencing other mental states, such as mental workload, mental stress, affective responses, attention, deception, preference, anticipation, suspicion, and frustration (Suzuki et al, 2008;Ayaz et al, 2012;Kreplin and Fairclough, 2013;Ding et al, 2014;Hirshfield et al, 20142014;Tupak et al, 2014;Arefi Shirvan et al, 2018;Numata et al, 2019). This indicates that increased Δ[HbO] values are not only an indication of fear responses but can also be driven by other psychological factors.…”

The neurovascular coupling in the attention during visual working memory

“…The use of EEG-fNIRS has paved the way for the use of ecologically valid settings to eliminate acoustic artifacts in fMRI scanning. Some EEG-fNIRS studies have focused on verbal WM [85, 86] or vWM by parallel analysis [87, 88]. Few studies have been performed to investigate attention during vWM.…”

Informing deep neural networks by multiscale principles of neuromodulatory systems

“…Experimental studies on neural correlates of attention have revealed biological underpinnings of different attentional forms, for example, visual, auditory, olfactory, and crossmodal attention. Humans [152,153] Near-infrared spectroscopy (NIRS) Seconds Hours, can be up to days cm (cortical area) Non-human primates, humans [154,155] Functional magnetic resonant imaging (fMRI)…”