Assessing the Driver’s Current Level of Working Memory Load with High Density Functional Near-infrared Spectroscopy: A Realistic Driving Simulator Study

Abstract

Cognitive overload or underload results in a decrease in human performance which may result in fatal incidents while driving. We envision that driver assistive systems which adapt their functionality to the driver’s cognitive state could be a promising approach to reduce road accidents due to human errors. This research attempts to predict variations of cognitive working memory load levels in a natural driving scenario with multiple parallel tasks and to reveal predictive brain areas. We used a modified version of the n-back task to induce five different working memory load levels (from 0-back up to 4-back) forcing the participants to continuously update, memorize, and recall the previous ‘n’ speed sequences and adjust their speed accordingly while they drove for approximately 60 min on a highway with concurrent traffic in a virtual reality driving simulator. We measured brain activation using multichannel whole head, high density functional near-infrared spectroscopy (fNIRS) and predicted working memory load level from the fNIRS data by combining multivariate lasso regression and cross-validation. This allowed us to predict variations in working memory load in a continuous time-resolved manner with mean Pearson correlations between induced and predicted working memory load over 15 participants of 0.61 [standard error (SE) 0.04] and a maximum of 0.8. Restricting the analysis to prefrontal sensors placed over the forehead reduced the mean correlation to 0.38 (SE 0.04), indicating additional information gained through whole head coverage. Moreover, working memory load predictions derived from peripheral heart rate parameters achieved much lower correlations (mean 0.21, SE 0.1). Importantly, whole head fNIRS sampling revealed increasing brain activation in bilateral inferior frontal and bilateral temporo-occipital brain areas with increasing working memory load levels suggesting that these areas are specifically involved in workload-related processing.

Mind over motor mapping: Driver response to changing vehicle dynamics

“…The parietal cortex also plays a role in updating motor plans based on visual feedback (Desmurget et al, 1999). Our results are consistent with previous fNIRS evidence that increased workload and attention demands during simulated driving are associated with increased activity in the PFC (Shimizu et al, 2013; Unni et al, 2017). Future studies will be required to tease apart the potentially overlapping cortical responses specific to visuomotor mapping and vehicle navigation (i.e., the baseline driving task) and the results of such studies could have important implications for the design of vehicle safety systems.…”

“…Characterizing and quantifying the neural circuitry underlying these processes has been a goal among cognitive neuroscientists using various neuroimaging techniques with varying complexity of task design. Functional MRI (fMRI), which requires participants to lie supine in a tube, has been used to examine neural activation in simulated driving and navigation paradigms (Calhoun et al, 2002; Unni, Ihme, Jipp, & Rieger, 2017; Walter et al, 2001), and to investigate fundamental components of driving behavior, such as visuospatial and visuomotor processing (Ng et al, 2000; Pollmann, & von Cramon, 2000) and attention (Beauchamp et al, 2001). Functional near infrared spectroscopy (fNIRS), a portable, noninvasive optical imaging technique, has been used to measure cortical activation during on-the-road driving (Yoshino et al, 2013) and in realistic, simulated driving tasks (Tsunashima and Yanagisawa, 2009; Xu et al, 2017).…”

A Novel Method for Classifying Driver Mental Workload Under Naturalistic Conditions With Information From Near-Infrared Spectroscopy

“…In conclusion, as previously suggested (Kopton and Kenning, 2014 ; Unni et al, 2017 ), simple NIRS has considerable potential for capturing driver mental workload, especially under naturalistic conditions.…”

Multisubject “Learning” for Mental Workload Classification Using Concurrent EEG, fNIRS, and Physiological Measures

“…It can be seen from Table 2 that fNIRS-alone classification accuracy is about 10% lower than EEG-alone classification. A recent fNIRS-based workload estimation study reports that using only the forehead optodes resulted in a much-reduced workload estimation accuracy compared to using optodes from the whole head (Unni et al, 2017). We speculated that by using whole head coverage, the fNIRS-alone and EEG+fNIRS performance can be much improved.…”