Robust Multimodal Cognitive Load Measurement (RMCLM)

Abstract

This report summarizes the important research activities, study results and research accomplishments out of the RMCLM project in the past year period. The objective of this project includes research of the fundamental issues related to the use of multiple input modalities and their fusion to enable robust and automatic cognitive load measurement (CLM) in the real world. Firstly, we carried out a further literature review on physiological measures of cognitive workload to include the recent advances of physiological measures of cognitive workload. In the meantime, we examined the use of various features (e.g. spectral and approximate entropies, wavelet-based complexity measures, correlation dimension, Hurst exponent) of electroencephalogram (EEG) signals to evaluate changes in working memory load during the performance of a cognitive task with varying difficulty/load levels. Eye based CLM was also studied. Three types of eye activity were investigated: pupillary response, blink, and eye movement (fixation and saccade). We further investigated the linguistic feature based CLM in this study and analyzed novel linguistic features as potential indices of cognitive load. All together, we had carried out CLM study of three unobtrusive modalities, namely EEG, eye activity, and linguistic feature based CLM, in the past year period. List of Publications Report Documentation PageForm Approved OMB No. Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. REPORT DATE APR 20132. REPORT TYPE This report summarizes the important research activities, study results and research accomplishments out of the RMCLM project in the past year period. The objective of this project includes research of the fundamental issues related to the use of multiple input modalities and their fusion to enable robust and automatic cognitive load measurement (CLM) in the real world. Firstly, we carried out a further literature review on physiological measures of cognitive workload to include the recent advances of physiological measures of cognitive workload. In the meantime, we examined the use of various features (e.g. spectral and approximate entropies, wavelet-based complexity measures, correlation dimension, Hurst exponent) of electroencephalogram (EEG) signals to evaluate changes ...

Multimodal behavioral and physiological signals as indicators of cognitive load

“…A great variety of cognitive load measuring techniques, from simple questionnaires to complex functional brain imaging [Whelan 2007] , have been developed to study cognitive load. Following are four primary methods comprising the current state of the art [Chen et al 2016]:…”

“…On the other hand, physiological measures may be affected by many other factors, such as engagement, fatigue, stress, and environment. Therefore, cognitive load measurement based on multiple modalities has been adopted to improve the robustness and accuracy [Chen et al 2016]. As shown in Figure 1-1, signals from various sensors and input devices can be collected during cognitive tasks.…”